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ABSTRACT 


The  Department  of  Defense  (DOD)  failed  in  its  previous  attempt  to  acquire  an 
enterprise  electronic  health  record  (EHR)  system.  The  earlier  program  was  plagued  with 
schedule  delays  and  cost  overruns,  which  caused  its  failure.  In  July  2015,  the  DOD’s 
Program  Executive  Office  awarded  a  $4.3  billion  contract  for  a  new  EHR  system  that 
was  below  cost  and  ahead  of  schedule.  The  objective  of  this  research  is  to  investigate  the 
key  reasons  why  the  DOD  has  succeeded  in  acquiring  its  most  recent  EHR. 

This  study  interviewed  nine  members  of  the  Program  Management  Office  (PMO) 
team  for  their  opinions  on  and  experiences  with  their  acquisition  and  management 
strategies  used  during  the  procurement.  The  research  showed  that  members  from  the 
program  management  to  the  program  executive  level  shared  commonalities  in 
management  styles  that  led  to  the  successful  acquisition  of  the  DOD’s  newest  EHR 
system.  The  research  identified  several  factors  key  to  the  program’s  success:  a  tailored 
acquisition  plan  that  allowed  the  PMO  to  directly  report  to  the  Under  Secretary  of 
Defense  for  Acquisition,  Technology  and  Logistics;  a  separation  between  the  medical 
communities  and  the  acquisition  team;  an  engagement  with  industry  early  in  the  process; 
and  a  motivated  leadership. 
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I. 


INTRODUCTION 


A.  PURPOSE 

In  February  2013,  the  U.S.  Department  of  Defense  (DOD)  and  U.S.  Department 
of  Veterans  Affairs  (VA)  ended  their  partnership  in  developing  a  joint  electronic  health 
record  (EHR)  that  would  allow  for  interoperability  of  data  between  the  two  departments. 
According  to  the  U.S.  Government  Accountability  Office  (2014),  the  abandoned  project 
wasted  four  years  and  roughly  $1  billion  between  the  DOD  and  VA.  In  July  2015,  the 
DOD  announced  it  had  purchased  a  new  EHR  system.  According  Garamone,  the  program 
management  office  (PMO)  responsible  for  the  acquisition  of  the  new  system  acquired  a 
commercial  product  for  $4.3  billion  in  only  two  years.  The  contract  award  of  the  new 
EHR  system  was  on  schedule  and  below  cost  (2015). 

The  purpose  of  this  study  is  to  examine  why  the  DOD  has  succeeded  in  acquiring 
its  most  recent  enterprise  EHR  system  in  record  time  at  a  significant  cost  savings  when 
the  previous  attempts  had  failed. 

B.  OVERVIEW 

The  DOD  has  a  long  history  of  attempting  to  acquire  an  EHR  for  the  longitudinal 
storage  and  access  of  healthcare  data  for  active  duty  service  members  and  their 
dependents.  In  the  1980s,  the  DOD  implemented  its  first  EHR,  the  Composite  Health 
Care  System  (CHCS),  which  initially  achieved  success.  However,  future  attempts  to 
upgrade  the  CHCS  system  resulted  in  numerous  acquisition  failures  (Government 
Accountability  Office  [GAO],  2002). 

Since  the  CHCS,  several  attempts  to  acquire  and  modernize  the  DOD’s  EHR  have 
been  plagued  with  cost  over-mns  and  delays  (GAO,  2010).  Several  reports  by  the 
Government  Accountability  Office  (GAO)  reveal  previous  EHR  acquisitions  were 
impacted  by  poor  project  planning,  management  weaknesses,  lack  of  governance, 
inadequate  accountability,  and  poor  oversight,  which  cost  taxpayers  billions  over  the  span 
of  10  to  15  years  (GAO,  2010). 
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In  July  2015,  the  Program  Executive  Office  (PEO)  of  the  Defense  Health 
Management  Systems  (DHMS)  awarded  a  $4.3  billion  contract  to  purchase  a  commercial 
off-the-shelf  (COTS)  EHR.  The  Defense  Healthcare  Management  System  Modernization 
(DHMSM)  program  did  not  incur  protests  from  the  competition  and  was  able  to  reduce 
the  cost  from  an  estimated  $13  billion  to  an  actual  cost  of  $8.3  billion  for  the  estimated 
10-year  life-cycle  of  the  implementation  (Garamone,  2015).  It  is  unclear  specifically 
what  the  PEO  DHMS  (hereafter  referred  to  as  PEO)  did  to  achieve  this  success. 

C.  RESEARCH  QUESTIONS 

This  research  examines  how  the  DOD’s  PEO  and  DHMSM  program  succeeded  in 
successfully  awarding  a  multi-billion-dollar  contract  for  a  DOD  electronic  health  record 
when  previous  attempts  to  acquire  an  EHR  had  failed. 

Related  research  questions  include  the  following: 

•  What  were  the  key  reasons  for  the  failures  of  previous  enterprise  EHR 
programs  within  the  DOD? 

•  What  obstacles  and  risks  did  the  program  office  face  when  implementing 
the  new  system? 

•  What  acquisition  and  managerial  strategies  directly  led  to  success? 

•  What  can  current  program  managers  of  major  software  acquisitions  learn 
from  the  DHMSM  program? 

D.  PROJECT  OBJECTIVES  AND  SCOPE 

The  primary  objective  of  this  research  is  to  investigate  the  key  reasons  why  the 
DOD  has  succeeded  in  acquiring  its  most  recent  enterprise  EHR  in  record  time  and  at  a 
significant  cost  savings.  More  specifically,  this  study  focuses  on  events  from  the  PEO 
and  the  DHMSM  program  office  from  its  creation  in  June  2013  to  the  new  contract  award 
in  July  2015.  A  secondary  objective  is  to  explore  how  and  why  previous  attempts  by  the 
DOD  to  acquire  an  EHR  failed  to  achieve  the  same  success.  A  final  goal  is  to  provide 
feedback  and  proposals  for  improvements  to  the  DOD’s  acquisition  framework  to  benefit 
other  major  acquisition  programs. 
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This  research  examines  the  acquisition  phase  for  the  programs  in  question.  This 
research  does  not  explore  the  implementation  or  deployment  of  the  systems.  In  Chapter 
II,  the  costs  and  other  implementation  concerns  are  documented  for  each  previous  DOD 
EHR  program.  This  is  to  compare  the  relative  effectiveness  of  their  corresponding 
acquisition  processes.  This  research  does  not  address  any  future  activities  or 
implementation  plans  from  the  PEO  or  the  DHMSM  program  management  offices. 

E.  METHODOLOGY 

This  study  used  interviews  with  key  people  from  the  PEO  and  the  DHMSM 
program  office.  These  people  had  direct  experience  with  and  decision  authority  in  the 
program.  The  interview  format  enabled  each  person  to  tell  his  or  her  story  directly  and  in 
detail,  and  by  integrating  the  stories  of  multiple  key  people,  this  research  was  able  to 
weave  together  a  rich  and  coherent  understanding  and  set  of  lessons  learned.  From  the 
PEO  office,  two  executives  were  interviewed.  From  the  DHMSM  program,  three 
program  managers  were  interviewed,  and  within  the  PEO  and  DHMSM  program,  four 
key  managers  were  interviewed. 

This  intent  of  this  section  is  to  show  the  research  organization  and  procedures. 

1.  Subjects 

Each  interview  subject  was  selected  based  on  his  or  her  position  and  influence  in 
the  program.  With  the  support  of  co-advisor  Mark  Krause,  who  is  familiar  with  the 
program  office  and  personnel,  a  cohort  of  12  individuals  was  identified  as  possible 
research  subjects.  A  total  of  nine  subjects  volunteered  to  participate  in  the  research. 
When  the  first  nine  subjects  were  selected,  the  researcher  did  not  pursue  or  consider  any 
additional  research  candidates.  Research  participants  were  checked  by  the  student 
researcher  against  the  PEO  and  DHMSM  organizational  charts  for  their  positions  and 
roles  during  the  timeframe  in  question.  The  researcher  also  submitted  the  list  of  potential 
participants  to  Mark  Krause  for  approval  before  providing  the  research  candidate  a 
consent  form. 
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2. 


Consent 


All  research  subjects  were  provided  a  consent  form  prior  to  their  participation  in 
the  research.  The  researcher  requested  that  the  subjects  read  and  understand  the  research 
consent  process  prior  to  signing  the  form.  If  the  subjects  agreed  to  the  conditions  of  the 
research,  they  signed  and  dated  the  consent  form  and  returned  it  to  the  researcher. 

3.  Interview  Procedures 

The  subjects  were  asked  to  answer  in  detail  five  questions  based  on  their 
experiences  or  observations  during  the  selected  timeframe  at  the  PEO  or  the  DHMSM. 
Interviews  lasted  30  minutes  or  less  and  were  conducted  either  face-to-face,  over  the 
telephone,  or  via  email. 

4.  Privacy 

The  Institutional  Review  Board  (IRB)  at  the  Naval  Postgraduate  School  (NPS) 
required  all  subjects  to  be  de- identified  after  the  interview  to  protect  their  privacy.  During 
the  writing  process,  the  research  subjects  are  referenced  with  pseudonyms  according  to 
the  positions  they  held  during  the  selected  timeframe.  Examples  include  program 
executive  1,  program  manager  1,  and  manager  1. 

5.  Interview  Questions 

The  interview  questions  were  derived  from  the  research  questions: 

1.  Were  you  involved  in  supporting  any  previous  DOD  EHR  related 
programs?  What  do  you  believe  were  some  of  the  key  reasons  for  the 
failure  of  these  past  EHR  programs? 

2.  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

3.  What  major  acquisition  strategies  were  implemented  to  overcome  these 

obstacles  and  risks? 

4.  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS  and  the 
DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the 
DHMSM  program? 
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5.  Based  on  your  experience  as  a  member  of  the  PEO  DHMS  team,  what  are 
the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  a  DOD 
PM  of  other  major  software  acquisition  programs? 


F.  RESEARCH  BENEFITS 

The  examination  of  the  current  PEO  and  DHMSM  program  provides  a  unique 
academic  opportunity  for  learning  how  organizational  behavior  affects  large  DOD 
acquisition  purchases.  In  addition,  the  lessons  learned  from  a  successful  program  can  be 
used  to  streamline  other  major  DOD  acquisition  projects  and  help  support  other  federal 
agencies.  When  the  DOD  has  the  opportunity  to  implement  lessons  learned,  it  can  be  a 
better  steward  of  the  American  taxpayers’  money,  provide  better  services  for  military 
members  and  their  dependents,  and  most  important,  potentially  save  lives. 

G.  ORGANIZATION  OF  RESEARCH 

Chapter  II  provides  a  history  of  the  electronic  health  record  in  the  United  States,  a 
review  of  the  DOD’s  previous  EHR  acquisitions,  as  well  as  literature  documenting 
implementation  barriers  and  lesson  learned.  Chapter  III  presents  the  data  from  the 
research  subjects’  interviews,  and  Chapter  IV  provides  an  analysis  of  the  findings. 
Finally,  Chapter  V  presents  the  researcher’s  conclusions  and  suggests  future  research 
opportunities. 
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II.  LITERATURE  REVIEW 


This  chapter  includes  a  brief  history  of  the  electronic  health  record  (EHR)  and 
recent  legislative  developments.  In  addition,  it  discusses  the  history  of  the  DOD  EHR,  as 
well  as  acquisitions  and  implementation  failures.  Moreover,  it  details  the  failed  attempt  of 
the  DOD  and  VA  to  jointly  build  an  EHR.  This  chapter  also  illuminates  the  challenges  of 
other  EHR  implementation  from  around  the  world.  Finally,  the  organizational  and 
reporting  structures  of  the  Program  Executive  Office  (PEO)  are  discussed.  The  literature 
review  serves  as  a  basis  to  understand  what  led  to  obstacles  for  the  PEO  and  DHMSM. 

A.  BACKGROUND 

In  April  2004,  President  George  W.  Bush  announced  his  goal  that  every  American 
have  an  electronic  health  record  by  2014  (White  House,  n.d.).  He  urged  coordination 
between  public  and  private  sectors  to  accelerate  adoption  of  health  information 
technology.  In  2009,  President  Barack  Obama  signed  the  Health  Information  Technology 
for  Economic  and  Clinical  Health  Act,  also  known  as  the  “HITECH  Act,”  with  the  goals 
of  advancing  the  meaningful  use  of  health  information  technology  (HIT),  improving  the 
quality  of  care,  and  supporting  the  further  development  of  EHRs.  In  addition  to 
meaningful  use,  the  act  incentivized  organizations  that  took  on  the  risk  of  building  an 
EHR  (Centers  for  Disease  Control  and  Prevention,  2016).  According  to  Tripathi  (2012), 
the  HITECH  Act  helped  create  commonality  in  basic  EHR  functions,  which  propelled 
industry  to  drive  the  health  information  infrastructure. 

1.  History  of  Electronic  Health  Record 

According  to  Ayatollahi,  Mirani,  and  Haghani  (2014),  “EHR  system  is  an 
information  system  that  helps  collect  the  health  information  of  individuals  from  birth  to 
death  and  provide  real-time  patient  health  records  to  support  clinicians  in  decision 
making.”  The  main  goal  of  implementing  an  EHR  is  to  improve  the  quality  of  care  by 
reducing  medical  errors  by  removing  paper  medical  records  and  by  increasing 
communication,  data  collection,  and  information  sharing  among  health-care  providers 
(Ayatollahi,  Mirani  &  Haghani,  2014). 
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Prior  to  the  1960s,  medical  documentation  was  primarily  paper  based.  During  the 
1960s,  electronic  systems  began  to  transform  paper-based  medical  records  into  EHRs. 
The  first  generation  held  basic  information  such  as  the  patient’s  name,  address,  date  of 
birth,  social  security  number,  and  insurance  information  in  a  computer  terminal  (Egyhazy 
&  Mukherji,  2004).  Egyhazy  and  Mukherji  (2004)  explained  that  these  systems  were 
dormant  and  not  user  friendly.  The  information  was  not  interoperable  with  other  systems. 
Another  difficulty  came  in  that,  in  order  to  access  information,  the  user  had  to  access  the 
computer  in  a  fixed  location.  Advances  in  networking  infrastructure  that  made  it  possible 
to  access  a  patient’s  medical  information  from  other  computer  platforms  came  in  the 
1970s  and  1980s  (Egyhazy  &  Mukherji,  2004).  This  led  to  the  development  of  the  DOD’s 
first  EHR  system,  the  Composite  Health  Care  System  (CHCS)  in  1985. 

2.  Military  Electronic  Health  Record 

An  EHR  can  play  a  vital  role  in  the  management  of  care  received  by  members  and 
their  dependents  over  long  periods  of  time.  Within  the  military,  a  member’s  health  record 
is  his  or  her  living  document  from  first  entry  into  military  services  to  veteran  status.  The 
electronic  record  must  be  available  to  health-care  providers  from  multiple  military 
treatment  facilities  in  the  continental  United  States  (CONUS),  outside  continental  United 
States  (OCONUS  or  overseas),  and  in  operational  environments  (Government 
Accountability  Office  [GAO],  2014).  The  large  scale  and  required  interoperability  creates 
a  challenging  objective  for  DOD  and  increases  risk  for  health-care  operations. 

According  to  the  Military  Health  System  (2016a.),  the  DOD’s  “EHR  is  used  by 
more  than  100,000  medical  clinicians  at  military  hospitals  and  clinics  worldwide.”  This 
EHR  is  used  by  the  military’s  medical  community  to  capture,  manage,  and  share  data  on 
care  provided  in  garrison  and  on  the  battlefield  (Military  Health  System  [MHS],  2016a). 
In  today’s  military  environment,  there  are  two  primary  EHR  systems  in  use:  the 
Composite  Health  Care  System  (CHCS)  and  the  Armed  Forces  Health  Longitudinal 
Technology  Application  (AHLTA).  Both  systems  are  enterprise-wide  information- 
management  systems  that  provide  secure  access  to  health  care  records  of  active  duty 
service  members,  retirees,  and  beneficiaries  (MHS,  2016a).  Both  systems  are  explained 
further  in  subsequent  sections. 
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The  U.S.  Department  of  Veterans  Affairs  (VA)  uses  an  internally  developed  EHR, 
commonly  known  as  the  “best-of-breed”  system.  The  program  is  called  the  Veterans 
Information  Systems  and  Technology  Architecture  (VISTA).  It  provides  an  integrated 
inpatient,  outpatient,  and  administrative  functions  within  the  EHR.  VISTA  supports  over 
1200  healthcare  sites,  and  it  is  one  of  the  largest  networks  in  the  United  States.  It 
provides  care  to  over  eight  million  veterans,  163  hospitals,  and  800  clinics  through  the 
United  States  (Department  of  Veterans  Affairs  [VA],  n.d.). 

B.  HISTORY  OF  DOD  EHR 

This  section  goes  into  a  brief  history  of  the  previous  DOD  EHR  acquisitions. 

1.  Composite  Health  Care  System 

In  May  1985,  the  DOD  tasked  the  assistant  secretary  of  defense  for  health  affairs 
to  acquire,  implement,  and  operate  a  health  care  computer  system.  The  total  estimated 
cost  was  between  $800  million  and  $1.1  billion,  and  the  plan  called  to  deploy  CHCS  to 
167  military  hospitals  and  577  clinics  between  1987  and  1995  (GAO,  1986). 

During  the  acquisition  of  CHCS,  numerous  errors  caused  delays  in  system 
development  contracting.  First,  the  DOD  miscommunicated  with  some  vendors  and 
allowed  for  unacceptable  proposals  to  continue  with  contract  actions  when  they  should 
have  been  removed  early  in  process,  which  increased  costs  and  delayed  the  schedule. 
Second,  the  DOD’s  disorganization  with  the  industry  proposals  caused  many  proposals  to 
go  unreviewed  (GAO,  1987).  Third,  the  DOD  failed  to  document  previous  discussions 
with  industry  that  led  to  prior  contract  agreements.  Finally,  the  DOD  did  not  consider  all 
cost  factors  at  the  beginning  of  the  project;  therefore,  costs  increased  by  $7  million  when 
new  proposals  were  added  following  discussions  with  industry  (GAO,  1987).  Overall,  the 
CHCS  system  achieved  what  it  was  developed  to  do:  provide  electronic  storage  for 
service  members’  health  information. 

2.  Composite  Health  Care  System  II 

In  1997,  in  an  effort  to  improve  the  CHCS  program,  the  Office  of  the  Assistant 
Secretary  of  Defense  for  Health  Affairs  (ASD  [HA])  directed  the  acquisition  of  a  new 
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system  to  replace  the  legacy  system.  The  new  system  was  named  the  Composite  Health 
Care  System  II,  later  known  as  the  Armed  Forces  Health  Longitudinal  Technology 
Application  (AHLTA).  AHLTA  was  developed  to  address  the  limitations  of  the  CHCS 
system.  According  to  Egyhazy  and  Mukherji  (2004),  a  major  limitation  was  the  program 
language  used  to  develop  the  CHCS  system.  It  was  created  with  Massachusetts  General 
Hospital  Utility  Multi-Programming  System  (MUMPS)  programming  language  that  did 
not  allow  for  other  modem  programming  languages  to  edit  the  code.  Second,  due  to  its 
unique  programming  language,  the  DOD  lacked  the  ability  to  contract  for  commercial 
venders  to  train,  maintain  and  evolve  the  system. 

The  AHLTA  system  was  developed  differently  from  CHCS.  The  plan  was  to 
create  an  architecture  with  highly  complex  automation  of  systems,  so  AHLTA  would 
comprise  many  other  independent  systems  (Egyhazy  and  Mukherji,  2004).  The 
acquisition  timeframe  was  expected  to  last  18  years  from  fiscal  year  (FY)  1997  to 
FY2014  (GAO,  2002).  According  to  a  report  by  the  DOD’s  Inspector  General  Report 
(DODIG,  2006),  AHLTA’ s  “full  operational  capability  had  been  delayed  by  four  years 
because  of  performance  issues.”  In  addition,  the  dental  module  had  to  be  redesigned  to 
meet  clinical  needs.  According  to  DODIG  (2006),  the  system’s  life  cycle  cost  increased 
$1  billion  (from  $4,023  to  $5,019  billion),  and  the  program  was  extended  for  another 
three  years.  In  2012,  the  DOD  cancelled  all  remaining  updates  to  AHLTA. 

3.  EHR  Way  Ahead 

According  to  the  GAO,  the  DOD  had  obligated  approximately  $2  billion  over  13 
years  to  AHLTA,  the  system  that  had  not  fully  met  its  mission  goals  (GAO,  2002).  The 
report  mentioned  that,  even  though  the  DOD  had  delivered  many  outpatient  modules,  many 
other  capabilities  originally  planned  for  deliver  had  been  scaled  back.  In  addition,  users 
frequently  complained  about  AHLTA’ s  significant  performance  issues.  To  address  these 
issues,  the  DOD  recommended  purchasing  a  new  commercial  system  named  “EHR  Way 
Ahead.”  Subsequently,  the  DOD  budgeted  $302  million  for  the  EHR  Way  Ahead  initiative 
(GAO,  2010).  The  DOD’s  goal  for  this  system  was  to  be  the  “department’s  comprehensive, 
real-time  health  record  for  service  members  and  their  families”  (GAO,  2014). 
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4.  Integrated  Electronic  Health  Record 

During  the  same  time  the  DOD  was  developing  its  plans  for  the  EHR  Way  Ahead 
program,  the  VA  also  attempted  to  modernize  its  VISTA  electronic  health  system.  The 
VA  had  spent  roughly  $600  million  from  2001  to  2007  on  projects  to  modernize  VISTA; 
however,  the  department  had  estimated  that  the  new  cost  would  be  closer  to  $11  billion 
with  a  completion  date  of  2018.  The  VA  terminated  the  VISTA  modernization  effort  in 
2010.  In  2010,  both  DOD  and  VA  secretaries  signed  an  executive  agreement  to  work 
together,  and,  in  2011,  a  new  charter  was  signed  to  start  the  Integrated  Electronic  Health 
Record  (iEHR)  effort.  This  combined  effort  was  expected  to  enable  the  VA  and  DOD  to 
align  resources  and  investments  with  common  business  needs  that  would  reduce  cost, 
increase  efficiencies,  and  create  an  interoperable  health  information  system  (GAO,  2014). 

After  years  of  developing  iEHR,  the  DOD  and  VA  announced  the  decision  to 
abandon  the  effort  in  February  2013.  This  decision  resulted  from  concerns  over  the 
program  facing  challenging  deadlines,  high  costs,  and  an  extended  timeline  to  deliver 
capabilities.  Based  on  these  assessments,  both  the  “DOD  and  VA  would  rely  on  separate 
systems  to  achieve  an  interoperable  EHR”  (GAO,  2014).  Possible  reasons  for  the  failures 
of  this  program  are  discussed  in  Chapters  III  and  IV,  in  which  program  managers  share 
opinions  on  why  this  program  failed. 

C.  BARRIERS  TO  IMPLEMENTING  AN  ELECTRONIC  HEALTH 

RECORD 

Several  studies  have  shown  the  complexity  in  adopting  and  implementing  an  EHR 
system  despite  the  potential  benefits.  This  literature  review  suggests  numerous 
organizations  that  have  attempted  EHR  implementation  have  encountered  road  blocks 
during  the  process.  These  studies  were  conducted  in  the  United  States  and  other  countries 
including  China,  Australia,  Iran,  and  the  United  Kingdom. 

Research  by  Standing  and  Cripps  (2015)  suggested  that  EHR  implementation 
downfalls  are  usually  a  result  of  complexities  in  the  project  scope,  resources,  decision¬ 
making  authority,  level  of  accountability,  expectation  by  stakeholders,  determination  of 
staff,  needs  of  the  clients,  and/or  level  of  resistance.  In  a  different  study,  researchers 
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Cucciniello,  Lapsley,  Nasi,  and  Pagliari  (2015)  documented  managerial  reasons  that 
hinder  EHR  implementation.  The  researchers  interviewed  numerous  clinicians  and 
documented  that  failed  relationships  among  stakeholders,  patients,  and  clinicians  can  lead 
to  potential  barriers.  In  addition,  the  adoption  of  an  EHR  that  affects  the  organization’s 
structure,  culture,  work  processes,  and  communication  channels  will  likely  face 
resistance  (Cucciniello  et  al.,  2015).  In  a  study  of  two  EHR  implementation  projects  in 
China,  Gao,  Xu,  Sorwar,  and  Croll  (2013)  identified  that  a  lack  of  governance, 
interoperability,  legislation,  and  skilled  professionals  all  contributed  to  challenges.  The 
researchers  agreed  that  a  strong  managerial  strategy  is  critical  for  a  successful  project. 

One  of  the  insights  from  Gao  et  al.  (2013)  and  Cucciniello  et  al.  (2015)  was  that 
people  and  their  relationships  are  vital  to  a  successful  EHR  implementation  strategy. 
According  to  Standing  and  Cripp,  the  critical  success  factors  that  led  a  successful  short¬ 
term  EHR  implementation  rely  on  system  focus  rather  than  strategic  focus.  In  other 
words,  “a  passionate  and  highly  committed  team  can  also  make  a  big  difference”  in  the 
success  of  the  project  (2015,  p.  83). 

D.  PEO  DEFENSE  HEALTH  MANAGEMENT  SYSTEM:  NEW  PROGRAM 

OFFICE 

This  section  covers  the  development  of  the  PEO  and  DHMSM  program  offices 
and  its  reporting  chain. 

1.  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics 

The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  (OUSD  [AT&L])  within  the  DOD  oversees  acquisitions,  research  and 
development,  advanced  technology,  and  logistics.  Figure  1  provides  a  representation  of  the 
undersecretaries  to  the  secretary  of  defense.  Overall,  the  USD  (AT&L)  supervises  all  DOD 
acquisition  projects  and  establishes  policies  for  the  procurement  of  goods  and  services, 
research  and  development,  testing  and  contract  administration  (OUSD  [AT&L],  n.  d.). 
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Figure  1.  Organization  Chart  for  the  Office  of  the  Secretary  of 

Defense  Principal  Staff  Assistants.  Adapted  from  Deputy  Chief 
Management  Officer  (2013). 


2.  New  Program  Executive  and  Program  Management  Office 

With  the  shift  in  priorities  after  the  iEHR  project,  both  the  DOD  and  VA 
announced  they  would  pursue  separate  paths  to  obtain  and  modernize  their  respective 
EHRs.  This  led  to  the  creation  of  the  Program  Executive  Office  (PEO)’s  Defense  Health 
Management  Systems  (DHMS).  The  PEO  manages  four  joint  programs  under  its 
portfolio.  The  PEO  mission  is  to  efficiently  improve  healthcare  for  active  duty  military 
members,  veterans,  and  beneficiaries  (Miller,  2015).  One  of  its  main  missions  is  to 
modernize  the  electronic  health  record  for  the  military  health  system  (MHS).  In  June 
2013,  the  PEO  established  the  Defense  Healthcare  Management  System  Modernization 
(DHMSM)  program  office.  The  DHMSM  priorities  have  been  to  select  and  award  a 
contract  for  a  state-of-the-art  EHR  that  will  replace  the  legacy  CHCS  and  AHLTA 
systems.  The  organizational  chart  for  the  PEO’s  DHMS  is  provided  in  Figure  2. 


Figure  2.  The  PEO’s  DHMS  Organizational  Chart.  Adapted  from  MHS  (2016c). 
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3.  Defense  Healthcare  Management  System  Modernization 

With  the  establishment  of  the  DHMSM,  the  program  office  has  a  clear  mission  to 
acquire,  integrate,  test,  and  deploy  a  modernized  EHR  system.  In  addition,  the  program 
will  unify  and  increase  accessibility  of  evidenced  based  healthcare  delivery  and  decision 
making  to  the  DOD  MHS.  The  DHMSM’ s  organizational  chart  is  shown  in  Figure  3. 
DHMSM  has  one  program  manager  and  a  deputy  program  manager.  In  addition,  it  has 
aligned  critical  departments  (assistant  program  managers)  directly  under  the  program 
manager. 


Figure  3.  The  DHMSM  Organization  Structure.  Adapted  from  MHS  (2014). 

4.  Reporting  Chain 

In  1986,  the  position  of  program  executive  officer  was  established;  he  or  she  will 
lead  the  PEO  program  office.  A  PEO  is  typically  an  executive  office  that  oversees 
multiple  program  offices  within  the  unique  service  component.  Program  offices  have  a 
strict  reporting  requirement  to  the  PEO.  Then,  the  PEO  reports  to  the  service  component 
acquisition  executives,  as  depicted  in  Figure  4.  According  to  the  Defense  Acquisition 
University  (2013b),  this  structure  provides  a  clear  line  of  authority  for  any  high-cost  and 
high-interest  programs  commonly  referred  to  as  Acquisition  Category  (ACAT)  III,  which 
referred  to  projects  valued  less  than  $185  million,  ACAT  II  ($835  million  or  more),  or 
ACAT  I  (projects  of  $2.79  billion  or  more). 


14 


In  2013,  as  the  planning  for  the  creation  of  the  PEO  program  office  was 
underway,  the  reporting  chain  to  USD  (AT&L)  was  modified.  PEO  no  longer  reports  to  a 
service  component  acquisition  executive;  the  director  of  the  program  now  reports  directly 
to  USD  (AT&L),  as  shown  in  Figure  5.  No  other  PEO  program  in  the  DOD  or  component 
services  has  this  waver  approved  to  directly  report  to  the  USD  (AT&L). 


Under  Secretary  of  Defense 
(AT&L) 


Assistant  Secretary’  of 
Equivalent 


General  Officers  SES 
Civilian 


COL/CAPT/Civilian 

Equivalent 


Figure  4.  Typical  Reporting  Chain  for  Acquisition  Programs.  Adapted  from 

Defense  Acquisition  University  (2013a). 
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Figure  5.  Reporting  Chain  of  Command  for  PEO  to  USD  (AT&L). 

Adapted  from  MHS  (2016b). 


5.  New  Contract  Award 

In  July  2015,  the  PEO  and  the  DHMSM  program  office  awarded  a  $4.1  billion 
contract  to  purchase  a  commercial-off-the-shelf  (COTS)  EHR  from  industry.  The  PEO 
streamlined  the  acquisition  process  to  select  and  procure  a  state-of-the-art  EHR  in  record 
time  with  no  protests  driving  down  the  expected  life  cycle  cost  from  $13  billion  to  $8.3 
billion.  This  thesis  analyzes  the  acquisition  strategy  and  management  oversight  used  by 
the  PEO  to  achieve  results  that  surpassed  all  previous  DOD  acquisition  attempts  for  an 
EHR  (Mazmanian,  2015). 
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III.  FINDINGS 


The  findings  presented  in  this  chapter  are  the  result  of  data  collected  by  the 
researcher  over  four  weeks  in  May  and  June  2016.  The  data  represented  in  this  chapter 
are  the  researcher’s  summary  of  the  subject  interviews  and  an  analysis  of  commonalities 
and  differences  among  the  responses. 

A.  ORGANIZATION  OF  FINDINGS 

At  the  conclusion  of  each  interview,  the  student  researcher  transcribed  the 
subject’s  words  into  an  electronic  format,  which  was  used  to  visually  organize 
commonalities  and  differences  in  the  responses.  The  student  researcher  has  grouped 
responses  in  this  chapter  based  on  the  individual’s  managerial  category:  executive, 
program  manager  (PM),  or  manager.  The  number  of  subjects  by  position  in  the  hierarchy 
is  tallied  in  Table  1.  Hereafter,  descriptive  pseudonyms  are  used  to  represent  each 
subject’s  position. 


Table  1.  Subject  Tally  by  Position 


Position 

Number  of  Subjects 

Executive 

2 

Program  Manager 

3 

Manager 

4 

Having  three  levels  of  the  PEO/DHMSM  hierarchy  is  important  for  determining 
successful  management  and  acquisition  strategies.  Each  level  brings  a  different  view  of 
what  is  important  or  what  challenges  the  program  faced.  Identifying  commonalities  and 
differences  among  levels  of  management  should  help  reveal  a  set  of  best  practices  for 
future  acquisitions  of  this  size. 
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B.  DATA  COLLECTION 


In  this  section,  the  commonalities  are  identified  among  the  subjects’  responses 
during  the  interviews.  Responses  are  compiled  into  tables,  and  the  frequency  of  answers 
are  tallied  according  to  the  individuals’  positions. 

1.  Question  la:  Were  You  Involved  in  Supporting  Any  Previous  DOD 
EHR-Related  Programs? 

Question  1  had  two  components.  The  first  question  asked  whether  the  member 
was  involved  in  any  of  the  DOD’s  previous  electronic  health  record  (EHR)  system 
acquisitions  or  implementations.  Table  2  shows  the  response  breakdown  by  member  and 
level. 


Table  2.  Question  1,  Part  A — Were  You  Involved  in  Supporting 
Any  Previous  DOD  EHR-Related  Programs? 


Executive 

PM 

Manager 

TOTAL 

Yes 

1 

2 

2 

5 

No 

1 

1 

2 

4 

TOTAL 

2 

3 

4 

2.  Question  lb:  What  Do  You  Believe  Were  Some  of  the  Key  Reasons  for 
the  Failure  of  These  Past  EHR  Programs? 

Part  two  of  Question  1  asked  the  subjects  to  elaborate  on  their  opinions  and/or 
experiences  surrounding  the  DOD’s  failed  EHR  projects.  The  researcher  recorded  the 
responses  from  subjects  who  were  involved  in  at  least  one  previous  DOD  EHR  project  (see 
Table  3)  and  those  who  were  not  involved  but  gave  their  opinions  anyway  (see  Table  4). 
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Table  3.  Question  1,  Part  B — What  Do  You  Believe  Were  Some  of  the  Key  Reasons  for 
the  Failure  of  These  Past  EHR  Programs?  “Yes”  Respondents 


Executive 

Poor  collaboration  across  organizations 

PM 

Attempt  to  develop  or  modify  a  product 

Unrealistic  project  scope 

Problematic  requirements 

Manager 

Different  internal  processes 

Lack  of  commitment  among  both  agencies  (DOD  and  VA) 

Too  many  stakeholders,  unwillingness  to  commit  on  a  way  forward 

Lack  of  single  authoritative  source 

Long  lead  times 

Multiple  governance  layers 

Lack  of  standard  framework  for  data  sharing 

Differences  in  organizational  functions 

Table  4.  Question  1,  Part  B — What  Do  You  Believe  Were  Some  of  the  Key  Reasons 
for  the  Failure  of  These  Past  EHR  Programs?  “No”  Respondents 


Executive 

Lack  of  stakeholder  buy-in  from  users 

Lack  of  understanding  of  requirements 

Differences  in  organizational  cultures 

PM 

Lack  of  acquisition  professionals  to  purchase  product 

Lack  of  buy-in  from  stakeholders 

Unsolidified  program  milestones 

Manager 

Lack  of  commitment 

Program  focus  on  home-grown  solutions 

3.  Question  2:  What  Were  the  Key  Obstacles  and  Risks  Faced  by  the 

PEO  DHMS  and  the  DHMSM  PMO  at  the  Beginning  of  the  DHMSM 
Program? 

Question  2  asked  the  subjects  to  identify  any  possible  obstacles  and/or  risks  that 
hindered  the  success  of  the  program  office.  The  researcher  attempted  to  highlight  the 
commonalities  and  differences  in  the  subjects’  responses  (see  Table  5). 
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Table  5.  Question  2 — What  Were  the  Key  Obstacles  and  Risks  Faced  by  the  PEO  DHMS 
and  the  DHMSM  PMO  at  the  Beginning  of  the  DHMSM  Program? 


Executive 

Lack  of  physical  organization 

Lack  of  right  people  for  the  job 

Conflicting  definitions  of  success  or  acquisition  strategy 

Communication  with  U.S.  Congress  (politics) 

Communications  with  industry 

Differences  in  system  requirements 

External  politics  from  past  EHR  failures 

PM 

Bad  press  from  U.S.  Congress 

Lack  of  governance  internal  to  the  program 

Lack  of  requirements 

Establishment  of  program  office 

Lack  of  funding 

Confusion  over  management  structure 

Previous  EHR  failures 

Top-heavy  bureaucracy 

Service  conflict  (Navy,  Army,  USMC,  Air  Force) 

Local  command  governance 

Unrealistic  requirements 

Manager 

Lack  of  formal  requirements 

Lack  of  empowered  central  leadership 

Aggressive  schedule  constraints 

Collaboration  across  multiple  organizations 

Negative  perceptions  from  previous  EHR  failure 

Lack  of  staffing  plans 

Lack  of  credible  cost  estimates 

Inability  to  secure  funding 

Lack  of  formal  requirements 
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4.  Question  3:  What  Major  Acquisition  Strategies  Were  Implemented  to 
Overcome  These  Obstacles  and  Risks? 

Question  3  is  a  continuation  of  the  previous  question.  It  asked  the  subjects  to 
identify  actual  strategies  used  that  positively  affected  the  program  office  and  led  to  the 
successful  award  of  the  new  EHR.  The  subjects’  responses  are  documented  in  Table  6. 
Question  3  leads  to  a  macro  view  focused  on  the  external  factors  that  affected  the  two 
program  offices. 


Table  6.  Question  3 — What  Major  Acquisition  Strategies  Were  Implemented  to 

Overcome  These  Obstacles  and  Risks? 


Executive 

Used  competition  effectively 

Engaged  industry  early  and  often 

Ensured  requirements  were  right 

Developed  a  tailored  and  flexible  contract  strategy 

Used  multiple  industry  days  to  communicate  to  industry 

Released  requests  for  information  (RFIs)  before  finalization 

PM 

Tailored  acquisition  documents  to  streamline  approval  process 

Reported  to  USD  (AT&L) 

Leveraged  lessons  learned  from  commercial  industry 

Created  no  artificial  requirements 

Focused  on  commonalities  of  a  uniform  health  care  system 

Allowed  for  an  aggressive  strategy 

Emphasized  streamlined  processes 

Manager 

Rebuilt  broken  relationships 

Overcame  staffing  obstacles  by  outsourcing  from  DHA  and  Space  and 

Naval  Warfare  Systems  Command  (SPAWAR) 

Reported  directly  to  USD  (AT&L) 

Challenged  the  status  quo 

Tailored  the  acquisition  plan 

Developed  requirements  through  draft  RFPs 

Partnered  with  industry 
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5.  Question  4:  How  Would  You  Describe  the  Managerial  Strategies  of 
the  PEO  DHMS  and  the  DHMSM  PMO,  and  How  Did  They 
Contribute  to  the  Success  of  the  DHMSM  Program? 

Question  4  asked  the  subjects  to  identify  specific  management  strategies  used  in 
either  program  office.  In  contrast  to  question  3,  this  question  offers  a  micro  view  focused 
on  the  management  strategy  within  the  two  groups.  The  subjects’  responses  are 
documented  in  Table  7. 


Table  7.  Question  4 — How  Would  You  Describe  the  Managerial  Strategies  of  the 
PEO  DHMS  and  the  DHMSM  PMO,  and  How  Did  They  Contribute  to  the 
Success  of  the  DHMSM  Program? 


Executive 

Communicated  with  all  levels 

Used  a  multi-dispensary  team 

Brought  in  acquisition  leadership 

Allowed  industry  more  time  to  plan  before  final  RFP 

PM 

Flattened  organization;  reduced  layers  and  bureaucracy 

Developed  meetings  with  a  purpose 

Created  relationships  with  requirements  community 

Developed  static  requirements 

Leveraged  previous  market  research 

Used  complete  transparency 

Used  aggressive  scheduling  with  milestones 

Published  acquisition  plans 

Met  with  industry  often 

Adopted  a  “hands-on”  leadership  approach 

Forced  people  to  make  decisions 

Engaged  the  functional  community  directly 

Manager 

Released  information  early 

Identified  the  right  skill  sets  and  staff  needed 

Ensured  the  right  community  was  part  of  the  functional  group 

Employed  leadership  with  COTS  acquisition  experience 
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Brought  in  highly  motivated  staff 

Used  a  tailored  acquisition  process 

Established  authority  to  proceed 

6.  Question  5:  Based  on  Your  Experience  as  a  Member  of  the  PEO 

DHMS  Team,  What  Are  the  Key  Lessons  Learned  and  Advice  You 
Can  Share  That  Can  Benefit  DOD  PM  of  Other  Major  Software 
Acquisition  Programs? 

Question  5  asked  the  subjects’  opinions  on  future  program-office  strategies  from 
which  other  large  acquisition  programs  can  learn.  The  subjects’  responses  are 
documented  in  Table  8. 


Table  8.  Question  5 — Based  on  Your  Experience  as  a  Member  of  the  PEO  DHMS 
Team,  What  Are  the  Key  Lessons  Learned  and  Advice  You  Can  Share  That 
Can  Benefit  DOD  PM  of  Other  Major  Software  Acquisition  Programs? 


Executive 

Under-promise  and  over-deliver 

Communicate 

Define  success/acquisition  strategy 

Adopt  the  enterprise  system 

Interact  early  with  industry 

Employ  change-management  and  training  early 

PM 

Collect  true  requirements 

Develop  a  solid  acquisition  strategy 

Adopt  a  COTS  product  with  commercial  capability 

Allow  enough  space  to  leverage  industry 

Be  descriptive  in  requirements 

Have  good  decisive  and  leadership 

Stay  within  organizational  functions 

Involve  functional  (medical)  community 

Force  functional  community  to  define  requirements 

Select  leaders  who  are  not  tied  to  the  medical  community 

Set-up  a  healthcare  systems  and  material  command  (SYSCOM) 
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Solidify  the  true  requirements 

Leverage  commonality — don’t  highlight  differences 

Manager 

Take  risks 

Streamline  the  acquisition  process 

Focus  on  teamwork;  communicate 

Establish  realistic  goals  and  timelines 

Create  positive  working  environments  that  motivate  employees 

Have  a  clear  understanding  of  requirements 

Initiate  early  interactions  with  industry 

Force  organizations  to  formalize  requirements 

Empower  a  central  leader  to  make  decisions  for  the  health-care 

enterprise 

Remove  customers  from  the  acquisition  decision;  leave  business 

decisions  to  acquisition  people 

C.  SUMMARY 

Responses  from  Question  1  highlight  the  cultural  differences  between  the  DOD 
and  the  VA,  which  led  to  the  iEHR  failure  to  mature  to  a  program  of  record.  Responses 
from  Question  2  expanded  the  uncertainties  in  standing  up  a  program  office.  The 
concerns  expressed  were  related  to  external  pressures  placed  on  the  program  office.  On 
the  other  hand,  in  answers  to  Questions  3  and  4,  it  appears  that  the  program  office  moved 
from  a  state  of  uncertainty  to  a  determined,  mission-focused  organization.  The 
management  strategy  and  requirement  gathering  were  important  during  this  phase. 
Question  5  yielded  some  unique  suggestions  for  improving  future  acquisitions  but 
reiterated  the  experiences  from  Questions  3  and  4.  Chapter  IV  further  analyzes  the  data 
presented  in  Chapter  III. 
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IV.  ANALYSIS  AND  DISCUSSION 


A.  ANALYSIS 

The  success  of  the  Defense  Healthcare  Management  System  Modernization 
(DHMSM)  and  Program  Executive  Office  (PEO)  is  broken  down  into  five  strategic 
improvements.  These  improvements  directly  led  to  the  successful  planning  and 
acquisition  of  the  DOD’s  EHR  system  in  July  2015.  These  five  improvements  include  the 
following:  true  requirements,  the  program  governance  structure,  hands-on  management, 
external  human  resources,  and  the  separation  between  acquisition  and  medical 
communities.  In  this  section,  we  use  the  subjects’  interview  responses  as  discussion 
points  to  the  five  strategic  improvements.  These  strategies  are  unique  to  the  PEO  and 
DHMSM  programs. 

1.  Requirement  Gathering 

In  software  engineering,  requirement  gathering  deals  with  establishing  the  needs 
of  the  stakeholders  that  the  software  fulfills.  During  the  acquisition  of  the  new  EHR,  the 
acquisition  team  was  in  charge  of  purchasing  COTS  software,  but  the  medical 
community  was  not  part  of  this  team.  It  maintained  a  separate  community,  known  as  a 
“functional  community.”  The  functional  community  provided  the  acquisition  team  with 
information  on  medical  necessities  for  the  new  software  to  perform. 

a.  True  Requirements 

Since  the  new  EHR  was  a  COTS  product,  it  was  critical  that  the  correct 
requirements  were  collected  prior  to  the  decision  to  purchase  the  software.  Program 
manager  2,  who  observed  the  DHMSM  program,  expressed  the  need  for  the  medical 
community  to  clearly  define  the  EHR’s  requirements:  “They  really  need  to  be  forced  to 
define  the  requirements.  The  tighter  you  get,  the  better  you’ll  be  able  to  buy  it  for  them. 
The  better  the  requirements,  the  better  the  acquisition  will  be”  (personal  communication, 
May  27,  2016).  Words  from  program  manager  2  support  the  finding  that  a  partnership 
between  the  acquisition  and  functional  communities  were  critical  for  success.  The 
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functional  community  performs  that  first  step  by  carefully  selecting  the  specific  tasks 
clinical  professionals  need  from  the  EHR  to  provide  medical  care  for  patients.  The 
acquisition  community  takes  those  specific  tasks  and  attempts  to  purchase  a  product  that 
meets  the  needs  of  the  functional  community. 

Specific  requirements  are  necessary  to  purchase  a  software  product;  however, 
sometimes  having  too  many  requirements  causes  delays  or  excess  costs  when  the 
software  is  purchased.  This  is  the  importance  of  having  true  requirements  provided  by 
the  functional  community.  Medical  professionals  have  the  tendency  to  add  requirements 
based  on  special  medical  processes,  which  add  to  the  complexity  of  an  EHR.  Program 
manager  1  suggested,  “We  should  not  create  artificial  requirements  based  on  the 
environment  we  operate  in.  We  should  make  sure  that  the  requirements  are  solidified  as 
true  requirements  and  differences  in  a  way  ...  solutions  are  presented”  (personal 
communication,  June  10,  2016).  Program  manager  1  echoes  the  findings  from  the  survey 
that  the  medical  community  needs  to  provide  the  acquisition  community  a  true  set  of 
requirements.  In  doing  so,  the  acquisition  community  better  negotiated  with  industry  to 
purchase  a  cost-efficient  product. 

Medical  fields  are  dynamic;  they  evolve  continually.  The  nature  of  medicine 
creates  a  challenge  when  a  group  of  clinical  professionals  are  tasked  with  developing 
requirements  for  an  EHR.  The  requirements  have  to  be  flexible  to  allow  for  changes  in 
medicine  and  the  specifics  of  medical  practice.  However,  the  acquisition  community 
needs  fixed  requirements.  Otherwise,  flexibility  or  indecision  results  in  lengthy,  costly 
evaluation  cycles.  Program  manager  1  commented  on  what  acquisition  teams  do  to  work 
around  this  problem: 

Acquisitions  are  static;  the  environments  that  they  intend  to  support  are 
dynamic.  So  you  have  to  build  contract  vehicles  that  are  flexible  and 
evolving  around  it.  We  know  that  the  requirements  will  not  be  static  at  the 
time  we  buy  the  product.  We  have  to  present  a  static  requirement  with  the 
framework  of  the  contract  with  the  ability  to  be  flexible  as  the  requirement 
changes.  That’s  a  fine  line  that  not  everyone  understands.  (Personal 
communication,  June  10,  2016) 
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Even  though  the  acquisition  process  and  the  requirements  need  to  be  static;  the  contract 
can  be  written  to  allow  flexibility.  In  the  new  EHR  contract,  additional  line  items  allowed 
for  future  modifications. 

b.  Partnership  with  Industry 

During  the  development  of  the  EHR’s  capabilities,  the  functional  community 
continued  to  disagree  over  what  true  requirements  should  be  used.  The  program  office 
successfully  used  industry  to  help  fill  in  the  gaps  for  the  medical  community.  Manager  1, 
who  was  involved  with  the  EHR  acquisition  selection  process,  discussed  using  industry 
to  fill  the  requirements  gap  to  support  the  functional  community: 

I  mentioned  that  they  [medical  community]  did  not  have  well-articulated 
formal  requirements,  but  what  the  program  office  did  was  they  partnered 
with  industry  and  utilized  an  iterative  draft  RFP  process  to  help  articulate 
requirements  that  industry  can  answer.  It  was  brilliant  because  the  services 
couldn’t  really  say  what  they  wanted,  but  they  had  an  idea  [of]  what  they 
wanted.  (Personal  communication.  May  26,  2016) 

The  DHMSM  program  office  frequently  used  industry  as  a  partner  to  build  its 
requirements.  What  DHMSM  did  was  unique  in  that  it  released  to  industry  a  request  for 
information  (RFI)  early  during  the  planning  process.  Even  though  the  RFI  was 
incomplete,  meaning  that  the  program  office  did  not  fully  know  its  final  requirements,  the 
office  released  it  to  industry  for  feedback.  The  DHMSM  program  office  continued  to 
release  many  other  RFIs  during  the  planning  phase.  This  led  to  the  development  of  a  rock 
solid  request  for  proposal  (RFP)  that  industry  could  fully  support.  Program  executive  2 
said  this  regarding  the  RFI  process: 

In  the  past,  we  in  government  waited  until  we  had  the  RFP  completed 
before  forwarding  to  industry.  This  time,  we  took  our  first  cut,  which  still 
needed  work  but  we  shared  it  with  industry.  We  got  their  feedback  and 
continued  to  get  internal  feedback  to  make  the  RFP  better.  I  think  that’s  a 
big  lesson  learned:  even  though  it’s  not  ready  for  prime  time,  share  that 
RFP  early.  (Personal  communication,  May  16,  2016) 

Sharing  the  RFP  early  and  often  allowed  the  program  office  to  increase  visibility 
with  industry  and  help  build  the  right  requirements  for  the  EHR  from  commercial  lessons 
learned.  Both  executives  and  managers  expressed  that  developing  requirements  to 
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populate  an  RFI  was  an  obstacle,  as  shown  in  Table  5.  However,  as  one  program 
executive  mentioned  in  Table  7,  the  use  of  industry  early  to  develop  the  requirements  was 
noted  as  a  best  practice  that  helped  the  government  receive  the  best  bids  at  very  good 
prices. 


2.  Governance 

As  discussed  in  Chapter  I,  no  other  program  executive  office  has  been  governed 
like  the  PEO  DHMS.  The  PEO  was  given  a  direct-reporting  status  to  the  USD  (AT&L), 
as  represented  in  Chapter  I,  Figure  5.  Normally,  a  PEO  falls  under  a  major  command  to 
govern  its  actions.  One  such  organization  is  the  Defense  Health  Agency  (DHA),  which 
reports  directly  to  the  secretary  of  defense.  Because  the  program  originated  in  Congress, 
it  was  high  profile,  and  Congress  controlled  the  cost.  Therefore,  selecting  the  right 
leadership  and  subordinate  was  key  to  governance.  Manager  2,  a  seasoned  PEO 
administrator,  described  how  USD  (AT&L),  the  head  of  PEO  DHMS,  was  crucial  to 
success: 

The  years  of  acquisition  experience  enabled  leadership  to  identify  the 
process  inherently  required  for  a  successful  COTS  acquisition.  They 
identified  the  right  skill  sets  and  staff  needed  to  meet  the  challenges  of 
developing  an  RFP  that  clearly  addressed  and  identified  the  requirements 
...  for  a  vendor  to  provide  a  COTS  solution.  Additionally,  through  their 
leadership,  they  ensured  the  right  communities  were  part  of  the  evaluation 
process  to  ensure  the  functional  requirements  would  be  met.  In  addition  to 
their  leadership,  folks  were  highly  motivated  to  support  the  mission  of 
obtaining  a  cradle-to-grave  EHR  for  our  service  members,  families,  and 
beneficiaries.  (Personal  communication,  May  25,  2016) 

One  smart  decision  leads  to  others.  U.S.  Congress  mandated  that  the  secretary  of 
defense  modernize  the  MHS’s  electronic  health  record.  Therefore,  the  DOD  created  a 
structure  for  the  PEO  to  align  directly  under  the  USD  (AT&L).  The  PEO  then  created  the 
DHMSM  program  office  to  directly  report  to  it.  Therefore,  the  PEO  had  only  one 
reporting  chain,  and  DHMSM  had  two  reporting  chains.  It  was  unprecedented  for  an 
AC  AT  I  program  of  its  size  to  have  minimal  governing  layers.  PEO  manager  2 
commented  on  the  benefits  of  directly  reporting  to  USD  (AT&L):  “Having  an  authority 
figure  such  as  the  secretary  of  defense  or  USD  (AT&L)  ...  influences  decisions  from 
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services’  figures  that  keep  your  program  moving.  The  petty  bureaucratic  fights  can  hold 
down  progress”  (personal  communication,  May  31,  2016).  This  hierarchy  was  vital  as  all 
services — Army,  Navy,  Marine  Corps,  and  Air  Force — report  directly  to  the  USD 
(AT&L)  for  their  procurements.  The  PEO  can  force  the  services  to  work  together  and 
come  to  terms  because  the  USD  (AT&L)  controls  funding  and  procurement.  This 
resolved  any  disagreement  among  the  services  and  kept  the  project  moving  forward. 
Program  manager  1  emphasized  how  this  tailored  acquisition  process  reduced 
bureaucracy: 

The  managerial  strategy  was  to  flatten  out  the  organization.  There  were 
too  many  layers.  When  you  are  in  a  compressed  time  line  to  deliver,  you 
can’t  have  a  lot  of  bureaucracy  at  lower  levels  where  decisions  are 
occurring,  so  USD  (AT&L)  fosters  that.  Access  to  leaders  that  make 
decisions  is  imperative.  (Personal  communication,  June  10,  2016). 

This  program  manager  noticed  the  difference  when  he  had  direct  access  to 
decisions  to  keep  the  program  moving.  As  shown  in  Table  7,  both  program  managers  and 
managers  mentioned  that  a  streamlined  reporting  chain  helped  them  obtain  decisions 
from  leadership  sooner.  Typically,  the  governance  of  the  services  and  services’  medical 
departments  can  hinder  the  progress.  However,  having  the  direct  support  of  the  PEO  and 
USD  (AT&L)  helped. 

3.  Management  Styles 

The  management  style  of  the  leaders  from  the  PEO  and  DHMSM  was  another 
important  factor  for  the  success  during  the  new  EHR  acquisition.  The  leadership  in  both 
program  offices  was  described  by  those  surveyed  as  very  aggressive  for  results.  The  PEO 
and  DHMSM  felt  pressure  to  deliver  a  product  by  a  specific  date  and  cost,  but  one  of  the 
risks  was  the  continued  indecisiveness  of  the  functional  community  in  laying  out 
requirements.  However,  program  manager  2,  who  observed  the  process,  mentioned  how 
the  aggressive  management  styles  worked  in  favor  of  moving  the  project  forward: 
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The  PEO  and  DHMSM  program  managers  were  very  hands-on  people. 

They  looked  to  engage  the  functional  community  directly  and  force  people 
to  make  decisions  and  get  them  on  board.  They  forced  people  to  have 
discussions  on  where  the  fiction  points  were  and  to  overcome  them.  You 
do  that  by  transparency  and  talking  to  people.  (Personal  communication, 

May  27,  2016) 

In  addition,  the  leadership  at  the  PEO  maintained  a  high  tempo  by  holding  weekly 
meetings  on  various  deliverables  from  the  DHMSM  program  office.  This  pace  ensured 
that  the  program  office  was  on  target  and  delivered  each  milestone  on  time.  Though  some 
might  view  this  style  of  management  as  micro-management,  it  worked. 

4.  Human  Resource  Sourcing 

Another  obstacle  that  the  DHMSM  program  faced  at  the  beginning  of  the 
acquisition  process  was  the  creation  of  a  physical  office.  As  the  survey  noted  from  the 
executive,  program  manager,  and  manager  levels,  one  consistent  concern  was  the 
establishment  of  a  program  office  that  would  be  responsible  for  purchasing  the  EHR 
system  (see  Table  5).  The  ability  to  find  the  right  staffing  for  the  positions  at  DHMSM 
proved  challenging  given  the  aggressive  timeline  for  the  PEO.  Manager  2,  who  was 
involved  with  the  PEO  from  the  beginning  of  the  program,  explained  the  obstacles  faced 
with  the  DHMSM  program: 

A  clear  plan  for  staffing  and  manning  the  projects  and  programs  was 
needed.  Then  obtaining  the  right  talent,  in  a  timely  manner,  to  execute  a 
plan  that  is  filled  with  challenges  ranging  from  long  hiring  times  for 
civilians[.]  [There  were]  service  members’  rotation  schedules  and  clearly 
obstacles  in  being  able  to  accomplish  the  work.  (Personal  communication, 

May  18,  2016) 

This  manager’s  worries  were  legitimate  because  for  the  federal  government, 
hiring  from  the  civilian  sector  could  take  many  months  of  advertisement,  recruitment,  and 
training  to  get  new  personnel  on  board.  Given  the  specific  knowledge  and  skills  required 
for  the  job  and  the  aggressive  acquisition  timeline,  this  would  prove  a  nearly  impossible 
task.  These  concerns  are  noted  in  Table  5,  wherein  both  the  executives  and  program 
managers  showed  concern  about  building  a  program  office  from  scratch.  Manager  2 
explained  that  the  “staffing  obstacles  were  overcome  with  the  agreement  between  DHA 
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and  SPAWAR — allowing  the  PEO  DHMS  to  hire  skilled  staff  from  the  working  capital 
funds”  (personal  communication,  May  18,  2016).  Essentially,  the  PEO  and  DHMSM 
cleverly  borrowed  personnel  from  other  program  offices  and  staffed  them  locally.  Even 
though  government  contractors  could  have  covered  the  majority  of  resource  gaps  within 
both  organizations,  government  expertise  was  critical  for  leading  and  holding  action 
accountable,  which  ultimately  led  to  the  success  of  the  EHR  system. 

5.  Maintaining  Functional  Community  Responsibilities 

The  separation  between  acquisition  and  medical  community  responsibilities  was 
also  critical.  The  DHMSM  program  was  set  up  with  acquisition  managers  leading  and 
medical  professionals  providing  guidance.  This  organizational  structure  was  different 
from  previous  EHRs,  wherein  the  medical  community  was  closely  involved  in  the 
acquisition  process.  The  development  of  the  altered  organizational  structure  started  with 
the  USD  (AT&L)  selecting  an  acquisition  professional  to  lead  the  PEO  program.  In  turn, 
the  PEO  selected  another  professional  with  extensive  acquisition-purchasing  and  COTS 
experience  to  lead  the  DHMSM  program.  Program  manager  2  observed  how  separating 
the  two  communities  affected  the  acquisition  process  positively: 

The  leaders  selected  to  lead  the  acquisition  should  not  be  directly  tied  to 
the  community  that  they  are  trying  to  service.  The  program  manager  of 
DHMSM  was  not  a  medical  doctor,  and  PEO  was  not  a  medical  guy.  I’m 
not  a  medical  guy;  I’m  an  acquisition  professional.  I  have  medical  people 
here  that  can  educate  me  and  advise  me  on  what  we  need  the  system  to  do. 

But  I’m  not  tied  to  a  particular  community...  [I]  wanted  to  get  the  best  for 
the  government  and  at  a  good  price.  There’s  no  loyalty  I  feel  to  a 
community.  Nothing  in  my  past  experience  influences  me  in  a  certain 
direction.  (Personal  communication,  May  27,  2016) 

Program  manager  2  echoes  the  findings  from  the  survey  that  medical 
professionals  may  present  a  conflict  of  interest  in  the  acquisition  process,  as  shown  in 
Chapter  III,  Tables  5,  7,  and  8.  The  goal  of  the  procurement  was  to  purchase  a  state-of- 
the-art  EHR  system  at  the  best  price  for  the  government.  The  individuals  best  suited  to 
negotiate  for  the  government  were  those  who  did  not  have  a  conflicting  interest  in  the 
procurement  and  understood  how  the  acquisition  process  works.  Medical  professionals 
are  not  trained  to  handle  large  acquisitions,  as  program  manager  1  explained: 
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Doctors  and  nurses  are  not  trained  to  do  acquisitions;  their  forte  is  to 
deliver  care  to  needy  patients.  You  can’t  treat  the  acquisition  community 
as  if  it’s  not  its  own  area  of  expertise.  You  must  be  an  expert  in  acquisition 
and  contracting  to  do  multi-billion-dollar  deals.  (Personal  communication, 

June  10,  2016) 

The  program  manager  expressed  that  it  took  experience  and  acquired  skills  to 
conduct  large-scale  acquisitions  properly.  The  program  manager  strongly  suggested  that 
the  acquisition  community  is  specially  trained  to  manage  these  type  of  projects.  There  is 
a  perception  in  the  medical  community  that  if  the  acquisition  is  a  medical  system  they 
should  have  a  lead  in  the  project.  Program  manager  1  explained  the  role  of  the  clinical 
and  acquisition  personnel: 

The  clinical  component  is  struggling  in  this  area.  They  got  to  get  an 
acquisition  professional  in  their  portfolios  and  then  allow  doctors  to 
practice  medicine,  which  they  are  allowed  to  do.  The  functional  piece  is 
the  doctors’  piece.  ...  The  acquisition  piece,  leave  it  to  the  acquisition 
professional.  (Personal  communication,  June  10,  2016) 

The  DHMSM  program  was  configured  to  have  two  acquisition  leaders  manage  it. 
The  medical  component  under  the  chief  medical  information  officer  (CMIO)  was  an 
assistant  program  manager  to  the  DHMSM  program  manager,  as  shown  in  Figure  3.  The 
CMIO  coordinated  information  gathering  among  the  functional  community  to  formulate 
the  necessary  requirements  for  the  program  office.  This  strategy  worked  for  DHMSM. 

B.  DISCUSSION 

In  this  section,  we  revisit  and  answer  the  research  questions  from  Chapter  I.  The 
interview  questions  were  derived  from  the  main  research  questions.  Data  collected  from 
Chapter  III  help  guide  the  answers  to  the  research  questions. 

1.  What  Were  the  Key  Reasons  for  the  Failures  of  the  Previous 
Enterprise  EHR  Programs  within  the  DOD? 

From  the  survey,  Question  2  presented  the  research  subjects’  responses.  The 
responses  were  based  on  experiences  and/or  insider  knowledge  (see  Table  3  and  4).  Some 
of  the  research  cohorts  had  had  numerous  experiences  with  health  IT  and  DOD 
acquisition  projects;  therefore,  they  could  speak  in  detail  about  the  subject. 
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Program  executive  2  suggested  that  the  previous  DOD  EHR  systems  were  not  all 
failures.  The  Composite  Health  Care  System  (CHCS)  and  Armed  Forces  Health 
Longitudinal  Technology  Application  (AHLTA)  programs  did  not  achieve  all  they  set  out 
to  do,  but  the  systems  did  move  the  military  health  system  toward  a  functional  EHR 
(personal  communication,  May  16,  2016).  However,  the  integrated  electronic  health 
record  (iEHR),  which  was  supposed  to  encourage  the  DOD  and  VA  collaboration,  failed 
to  achieve  its  mission.  This  section  discusses  the  opinions  of  why  the  iEHR  failed  and 
why  VA  and  DOD  decide  to  go  separate  directions. 

a.  Differences  in  Organizational  Culture 

Among  those  interviewed  who  direct  exposure  with  the  previous  EHR, 
experienced  had  shared  that  both  organizations  had  difficulty  collaborating  due  to 
difference  in  internal  process  (see  Table  3).  One  of  the  differences  was  the  inability  to 
decide  on  system  requirements.  During  this  time,  the  VA  wanted  to  update  its 
homegrown  Vista  solution  while  the  DOD  wanted  to  purchase  a  new  COTS  system.  This 
created  problematic  requirements,  which  developed  into  an  unrealistic  project  scope.  In 
addition,  there  were  many  stakeholders  from  medical  communities  from  both 
departments  who  added  to  the  requirements.  The  iEHR  program  could  not  move  forward 
due  to  the  levels  of  bureaucracy,  which  created  longer  than  normal  lead  times. 

Argumentatively,  the  iEHR  program  was  destined  to  fail  due  to  its  size  and 
complexity.  The  program  office  and  its  leadership  were  not  assertive  in  maintaining 
control  of  the  complex  system;  therefore,  the  program  lost  track  of  its  goals. 

2.  What  Obstacles  and  Risks  Did  the  Program  Office  Face  When 
Implementing  the  New  System? 

This  section  aligned  with  Question  2,  where  it  asked  the  research  subjects  to 
express  their  experiences  or  opinions  of  what  risks  DHMSM  faced  prior  to  the 
acquisition  and  implementation  of  the  new  DOD  EHR  system  (see  Table  5) 
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a. 


Previous  Failures 


Executives,  program  managers,  and  managers  all  echoed  that  the  iEHR’s  previous 
failure  had  a  profound  effect  during  the  early  stages  of  the  program.  The  negative 
publicity  received  from  the  iEHR  and  other  past  EHR  systems  had  many  individuals 
doubting  whether  the  DOD  would  be  able  to  procure  a  new  system.  The  negative 
publicity  affected  future  funding  and  working  relationships.  According  to  manager  2,  the 
new  PEO  head  worked  hard  to  reestablish  trust  within  the  DOD  and  its  stakeholders.  The 
PEO  actions  allowed  the  PEO  and  DHMSM  program  to  get  on  its  way  (personal 
communication.  May  25,  2016). 

b.  Uncertainty 

Executives,  program  managers,  and  managers  all  exhibited  concerns  over  the 
many  uncertainties  during  the  creation  of  the  DHMSM  program  office.  The  majority  of 
managers’  uncertainties  were  tied  to  standing  up  the  program  office  within  the  aggressive 
timeline.  Many  managers  noted  that  staffing,  confusion  over  the  management  structure, 
and  a  lack  of  formal  requirements  were  major  concerns.  The  executives  were  overly 
concerned  with  the  conflicting  acquisition  strategy  and  communication  with  industry, 
whereas  the  program  managers  were  concerned  about  service  interactions  and  the 
confusion  over  the  management  structure. 

3.  What  Acquisition  and  Managerial  Strategies  Directly  Led  to  Success? 

This  section  used  survey  responses  from  Questions  3  and  4.  Most  certainly,  the 
tailored  acquisition  that  allowed  the  PEO  to  report  directly  to  the  USD  (AT&L)  was  a 
major  factor;  however,  there  were  other  strategies  that  may  have  contributed  to  the 
success. 


a.  Industry  Communication 

Executives,  program  managers,  and  managers  emphasized  the  importance  of  early 
communication  with  industry  in  helping  with  the  requirements  collection  and 
development.  The  DHMSM  program  used  requests  for  information  (RFI)  effectively  to 
communicate  with  industry.  A  typical  program  office  would  spend  excess  amount  of  time 
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to  ensure  that  the  RFI  would  be  contain  complete  and  correct  information  before  it  is 
shared  with  industry.  This  method  can  be  ineffective,  as  industry  may  return  the  request 
back  to  DOD  as  something  impossible  to  do.  Early  communication  means  providing 
industry  limited  feedback  (requirements)  on  what  they  need,  then  the  commercial  sector 
will  be  able  to  fill  in  the  blanks  on  what  it  can  provide  to  the  federal  government.  The 
federal  government  then  takes  the  suggestions  by  industry  and  continues  to  build  on  its 
requirements.  Having  industry  tell  the  government  what  it  can  provide  early  is  extremely 
effective  because  the  government  is  trying  to  leverage  a  COTS  product.  The  government 
should  not  develop  requirements  that  industry  cannot  meet,  and  early  communications 
helps  reduce  that  risk  by  allowing  the  commercial  sector  do  the  market  research  for  the 
DOD. 


4.  What  Can  Current  Program  Managers  of  Major  Software 
Acquisition  Learn  from  the  DHMSM  Program? 

This  section  refers  to  Question  5  from  the  survey.  There  were  numerous 
suggestions  by  the  research  subjects  on  possible  lessons  learned.  The  commonalities 
among  the  executives,  program  managers,  and  managers  were  the  early  interaction  with 
industry  and  properly  adopting  the  COTS  product.  The  commonalities  between  program 
managers  and  mangers  were  decisive  leadership,  formalize  requirements,  streamline 
acquisition  process,  and  separation  between  acquisition  and  functional  communities. 

C.  SUMMARY 

In  summary,  the  analysis  section  provided  five  improvement  strategies  that  were 
critical  to  the  success  of  the  DHMSM  and  PEO  efforts  to  procure  the  DOD’s  new  EHR. 
The  governance  structure  provided  PEO  top  cover  to  manage  the  services’  expectations 
and  future  funds.  Without  the  support  from  the  USD  (AT&L)  for  the  PEO,  the  program 
could  have  fallen  behind  schedule.  The  collection  of  true  requirements  ensured  that  the 
DHMSM  program  purchased  an  EHR  system  that  was  state-of-the-art  and  cost  effective. 
Sourcing  from  other  government  organizations  helped  support  the  PEO  and  DHMSM 
programs  to  ensure  the  right  subject  matter  experts  filled  critical  resource  gaps  quickly. 
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The  separation  between  the  acquisition  and  medical  communities  ensured  that  conflicting 
interests  did  not  add  to  the  bureaucracy  and  that  personnel  focused  on  project  objects. 

The  leaders’  management  styles  allowed  all  of  these  strategies  to  happen.  The 
hands  on  leadership  of  the  PEO  and  DHMSM  programs  ensured  that  the  requirements, 
resources,  and  governance  were  established,  functional,  and  operational.  For  major 
information  technology  (IT)  acquisitions,  strong,  engaged,  and  aggressive  leadership  is 
needed  with  the  addition  to  political  top  cover. 

This  chapter  revisited  the  research  subjects’  responses  from  the  survey.  Failures 
of  a  program  office  generally  stem  from  numerous  factors,  but  the  subjects’  experiences 
linked  the  failure  of  the  iEHR  to  differences  between  the  two  departments’  (i.e.,  the  DOD 
and  VA)  cultures  and  the  inability  for  leadership  to  bring  the  requirements  toward  a 
common  goal.  The  challenges  the  PEO  and  DHMSM  largely  overcame  the  history  of  past 
EHR  failures  and  the  negative  perspectives  of  stakeholders.  This  resulted  in  many  project 
uncertainties.  Overall,  the  use  of  industry  as  a  partner  helped  ensure  the  program  office 
procured  a  state-of-the-art  system  at  a  very  good  price  for  the  government. 
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V.  CONCLUSION  AND  RECOMMENDATIONS 


This  chapter  summarizes  the  importance  of  this  thesis  and  discusses  the  data  used 
to  conduct  the  study.  In  addition,  the  chapter  contains  conclusions  drawn  from  the 
analysis  and  discussion  in  Chapter  IV.  Finally,  the  thesis  ends  with  recommendations  and 
suggestions  for  future  studies. 

A.  CONCLUSION 

The  information  presented  in  this  research  can  be  used  as  a  lesson  learned  for 
future  large  DOD  or  federal  acquisition  programs.  Moreover,  the  information  collected  is 
extremely  valuable  to  the  acquisition  community  supporting  medical  systems.  Due  to  the 
history  of  the  DOD’s  EHR  programs,  the  data  collected  in  this  research  can  serve  as 
building  blocks  for  improving  future  medical  acquisition  processes  as  well  as  all  major 
acquisition  programs.  The  ramifications  of  not  implementing  these  best  practices  could 
result  in  repeating  previous  failures. 

Generally,  failure  in  IT  systems  is  not  due  to  the  technology  itself;  it  is  due  to  the 
inability  of  the  users  to  train  fully  and  implement  the  system  into  their  daily  work  flows 
(Cripps  &  Standing,  2015).  In  program  management,  the  successful  acquisition  of  a 
system  requires  motivated  leadership  and  knowledgeable  staff  members  to  navigate 
obstacles  effectively  and  develop  management  strategies  to  succeed.  In  the  case  of  the 
DOD’s  EHR  system,  negative  press  and  congressional  mandates  had  already  created  an 
obstacle  for  the  PEO  even  prior  to  the  start  of  the  program.  Decisive  leadership 
understood  what  it  needed  to  make  the  process  work  effectively,  and  it  did  something  that 
went  against  the  grain  of  previous  acquisition  projects.  Implementing  a  tailored 
acquisition  process  for  the  PEO  and  DHMSM  programs  reduced  the  amount  of  red  tape 
for  critical  decisions.  Lessons  learned  from  the  past  ensured  that  leadership  placed  the 
skilled  acquisition  program-managers  needed  to  lead  the  procurement  and  to  keep  the 
functional  community  engaged — while  not  slowing  down  the  process.  Instead  of 
alienating  the  commercial  sector,  leadership  used  industry  effectively  as  a  partner.  This 
was  shown  to  be  a  best  practice  as  the  commercial  sector  gave  its  input  to  build  the 
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DOD’s  EHR,  played  a  part  in  the  process,  and  equally  marketed  products.  When  the 
contract  award  was  announced,  there  was  no  protest  from  industry,  and  this  is  very 
unusual  in  a  large  dollar  contract.  The  following  are  recommendations  drawn  from  this 
research  for  future  program  offices. 

1.  Create  a  Tailored  Acquisitions  Process 

Reduce  the  amount  of  red  tape  for  program  managers  to  get  critical  information 
approved  by  the  stakeholders.  The  majority  of  programs  suffer  the  additional  burden  of 
long  waiting  periods  before  a  key  decision  can  be  made.  Allow  the  program  manager 
access  to  executives  who  are  able  to  influence  the  decisions  of  multiple  stakeholders. 
Streamlining  the  acquisition  process  should  also  include  the  managers’  access  to  the 
program  managers  for  decisions  as  well. 

2.  Separate  Acquisition  and  Functional  Communities 

Traditionally,  medical  professionals  serve  an  important  and  influential  role  in  the 
program  management  of  a  health  information  technology  system.  This  could  possibly 
create  a  conflict  of  interest  wherein  the  medical  professionals  may  unintentionally  add 
requirements  that  are  unnecessary  for  an  enterprise  system.  Separating  the  roles — where 
acquisitions  is  in  charge  of  procurement  and  the  functional  community  advises — helps 
reduce  the  risk  of  adding  requirements  that  are  not  needed. 

3.  Create  Partnerships  with  Industry 

Normally,  the  federal  government  waits  until  it  has  a  complete  proposal  with  all 
requirements  before  sharing  any  information  with  industry.  With  the  DHMSM  program, 
information  was  shared  during  the  request-for-information  stages  before  all  requirements 
were  gathered.  According  to  executive  2,  members  of  industry  appreciate  the  early 
notification  because  it  allows  them  to  gather  their  teams  and  begin  planning  before  the  final 
request  for  proposal.  Executive  2  reiterated  that  in  many  instances,  industry  flies  blind  as  to 
what  the  government  needs.  Industry  needs  a  lot  of  lead  time  to  prepare  for  bidding  on 
large  programs.  The  federal  government  will  benefit  from  the  extra  lead  time  for  industry 
because  it  yields  more  competitive  bids  (personal  communication,  May  16,  2016). 
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4.  Motivated  Leadership 

For  COTS  procurement,  a  knowledgeable  leadership  in  acquisition  processes  with 
hands-on  program  governance  can  help  move  the  system  purchase  forward  and  keep  it  on 
track.  Leadership  needs  to  engage  the  functional  community  directly.  The  PEO  head 
should  engage  the  functional  community  at  the  highest  levels  with  the  support  of  the 
USD  (AT&L),  forcing  services  to  collaborate  and  buy-in  to  the  system.  The  DHMSM 
program  manager  should  aggressively  engage  the  functional  communities’  committees  to 
provide  true  requirements  to  the  acquisition  team  in  a  timely  manner.  These  leadership 
styles  were  able  to  bridge  the  two  communities  by  ensuring  that  difficult  decisions  were 
made  without  continued  friction,  a  critical  piece  to  the  success  of  both  programs.  The 
aggressive  leadership  displayed  during  PEO  and  DHMSM  programs  was  successful  in 
procuring  a  system;  however,  such  a  strategy  may  not  be  as  effective  for  sustainment 
efforts. 

B.  FUTURE  STUDY 

This  study  provides  an  examination  of  the  successful  strategies  the  PEO  and 
DHMSM  used  during  the  acquisition  of  the  DOD’s  new  EHR.  It  serves  to  identify  what 
entities  may  influence  a  successful  acquisition  program-management  office.  Referring  to 
Table  8  in  Chapter  III,  the  survey  responses  may  offer  a  guide  to  determine  future 
studies.  The  following  are  recommended  areas  for  future  study. 

1.  Examine  the  development  of  health  systems  and  material  commands. 

2.  Examine  the  improvement  for  standardizing  separating  acquisition  and 
functional  activities  for  large  program  projects. 

3.  Examine  the  impact  of  leveraging  the  use  of  the  commercial  industry  early 
during  the  acquisition  process. 

4.  Examine  implementing  a  streamlined  reporting  chain  for  major  acquisition 
programs  to  reduce  bureaucracy. 

5.  Examine  how  organizations  may  codify  and  articulate  their  requirement 
collection  to  result  in  a  successful  system. 
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APPENDIX  A.  PROGRAM  INTERVIEWS— PROGRAM 

EXECUTIVES 


The  following  are  excerpts  from  telephone  interviews  from  the  program 
executives.  Personal  identifiable  information  has  been  removed  to  protect  the  subjects’ 
privacy. 

Executive  1 

Interviewed  on  June  8,  2016 

Interview  conducted  via  telephone 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related 
programs?  What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past 
EHR  programs? 

•  No,  have  not  been  involved  in  previous  DOD  EHR  programs. 

•  Possible  lack  of  understanding  of  requirements. 

•  Possible  lack  of  stakeholder  buy-in  from  users. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

•  Lacking  physical  organization  then  had  to  stand  up  organization. 

•  Finding  the  right  people  for  the  job. 

•  Defining  what  success  looks  like  and  pulling  together  the  acquisition 
strategy. 

•  Communications  with  congress  and  services. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome 
these  obstacles  and  risks? 

•  Effectively  used  competition  and  engaged  industry  early  and  often. 

•  Industry  understood  requirement  early. 

•  Ensure  the  requirements  are  right  and  build  on  them. 

•  Had  DHA  and  services  spearhead  requirements  development. 

•  Developed  contract  strategy  that  was  tailored  and  flexible. 
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Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS 
and  the  DHMSM  PM,  and  how  did  they  contribute  to  the  success  of  the  DHMSM 
program? 

•  Communicate  with  people  with  what  is  happening. 

•  Use  a  multi-dispensary  team.  Use  people  with  all  kinds  of  technical 
abilities. 

•  Ensure  an  open  process  on  [how]  PEO  is  executing  the  program.  Held 
monthly  reviews  and  invited  stakeholders. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team, 
what  are  the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PMO  of 
other  major  software  acquisition  programs? 

•  Must  communicate. 

•  Under  promise  and  over  deliver. 

•  Must  adopt  the  enterprise  system. 

•  Must  have  change  management  and  training  for  user  adoption. 


Executive  2 

Interviewed  on  May  16,  2016 

Interview  conducted  via  telephone 


Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related 
programs?  What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past 
EHR  programs? 

•  Yes,  was  involved  with  ALHTA,  iEHR  and  MHS  Genesis. 

•  CHCS  was  not  a  failure,  it  moved  MHS  forward.  ALHTA  was  also  not  a 
failure,  it  did  not  achieve  everything  is  was  set  out  to  achieve,  as  several 
capability  blocks  were  not  delivered.  However,  it  moved  ambulatory  care 
and  documentation  forward,  [and]  moved  electronic  health  repository 
forward,  first  time  any  physician  can  see  a  health  record  at  any  part  of  the 
world.  In  that  regard,  it  was  a  success  but  it  did  not  achieve  everything  it 
set  out  to  do. 

•  iEHR  was  different.  DOD  and  VA  wanted  the  next  generation  electronic 
health  record  to  be  for  both  DOD  and  VA.  VA  and  DOD  had  different 
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views  on  how  the  EHR  should  proceed  because  DOD  looks  at 
Commercial  off-shelve  (COTS)  but  VA  had  been  a  pioneer  with  electronic 
health  record  development  but  has  fallen  behind  commercial  EHR  system 
capabilities. 

•  Commercial  capabilities  have  over  taken  Vista  capabilities.  DOD  made  a 
compromise  to  try  to  work  with  the  VA  on  iEHR,  instead  of  going  with 
the  best  of  suite  choice,  they  decided  to  with  VA  and  go  with  a  best  of 
breed  approach  with  the  iEHR.  There  were  reports  of  cost  overruns  but 
there  wasn’t  really  cost  over  runs  b/c  there  wasn’t  a  contract  for  iEHR  yet. 
The  program  was  stopped  before  the  best  of  breed  occurred. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

•  We  were  coming  out  of  iEHR  and  there  were  still  a  lot  of  people  that 
thought  we  should  still  continue  to  work  with  VA  and  use  their  VISTA 
system.  But  we  believed  that  VISTA  was  then  behind  the  market  and 
would  be  costly  to  upgrade  and  maintain  over  time.  DOD  decided  that 
VISTA  wasn’t  an  option.  That  was  an  obstacle  because  there  is  alot  of 
politics  involved. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome 
these  obstacles  and  risks? 

•  Multiple  RFIs  in  areas  we  had  questions  on  certain  aspects  of  the  EHR 
market  places.  So  we  did  targeted  RFIs  and  draft  RFPs  and  the  industry 
days. 

•  From  the  30  years  of  experience,  for  the  large  scale  systems  and  middle 
scale  systems,  this  is  the  best  interaction  with  industry  I’ve  seen,  to  get  the 
best  bids  at  a  very  good  price  to  meet  the  needs. 

•  In  the  past,  we  in  government  waited  until  we  had  the  RFP  completed 
before  forwarding  to  industry.  This  time,  we  took  our  first  cut  which  still 
needed  work  but  we  shared  it  with  industry.  We  got  their  feedback  and 
continued  to  get  internal  feedback  to  make  the  RFP  better.  I  think  that’s  a 
big  lesson  learned,  even  though  it’s  not  ready  for  prime  time-share  that 
RFP  early. 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS 
and  the  DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM 
program? 

•  The  strategy  that  [PEO  and  DHMSM  heads]  brought  with  them  combined 
with  knowledge  of  the  folks  that  lived  with  the  iEHR  and  ALHTA/CHCS 
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enabled  us  to  put  together  an  excellent  acquisition.  I  think  that  success 
when  you  combine  the  history  lessons  learned  with  the  new  thinking  on 
how  to  interact  with  industry  really  led  to  the  success. 

•  Industry  appreciates  the  draft  RFP  because  it  allows  them  to  get  ready 
their  teams  and  begin  planning  before  the  final  RFP. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team, 
what  are  the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of 
other  major  software  acquisition  programs? 

•  Inaction  with  industry.  A  lot  of  times,  industry  is  flying  blind  on  what  the 
government  really  needs.  Industry  needs  a  lot  of  lead  time  to  prepare  for 
any  large  program,  to  get  their  teaming  right,  and  to  do  a  competitive  bid. 
There’s  a  lot  of  work  that  needs  to  be  done  on  the  industry  side.  Putting 
out  RFPs,  [and]  RFIs,  not  only  allows  industry  the  ability  to  provide  input 
to  the  whole  process  but  also  allow  them  to  do  work  to  get  their  team 
together  [and]  to  prepare  to  bid  on  the  RFP  when  it  comes  out.  The  more 
time  you  give  industry,  the  better  off  they  will  be  to  offer  a  completive  and 
good  bid  and  it  totally  benefits  the  government  when  they  can  do  that  even 
though  you  are  not  ready  for  prime  time  with  your  RFP  yet.  So  being  able 
to  do  that  early  communication  is  tremendous  for  these  companies  because 
they  are  literally  flying  blind  when  stuff  comes  out  from  the  gov’t  as  a 
surprise. 
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APPENDIX  B.  PROGRAM  INTERVIEWS— PROGRAM 

MANAGERS 

The  following  are  excerpts  from  in-person  interviews  from  the  program 
managers.  Personal  identifiable  information  has  been  removed  to  protect  the  subjects’ 
privacy. 

Program  Manager  1 
June  10,  2016 

Interview  conducted  via  face-to-face 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related 
programs?  What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past 
EHR  programs? 

•  No,  was  not  part  of  any  previous  EHR  programs. 

•  Programs  in  the  past  lack  adequate  market  research,  requirements  or 
technical  agreement  from  respective  communities. 

•  Programs  in  the  past  lack  of  acquisition  professionals  and  team  to  acquire 
the  product. 

•  Programs  in  the  past  had  milestones  that  were  not  solidified. 

•  Programs  in  the  past  lack  appropriate  buy-in  from  stakeholders. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

•  Must  develop  the  ability  to  train  the  users  including  plans  for  maintenance, 
testing  and  deployment  of  the  system. 

•  Local  command  governance. 

•  Attempts  to  inhibit  the  commercial  product  with  unrealistic  requirements. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome 
these  obstacles  and  risks? 

•  Leveraging  what  has  been  done  in  the  commercial  environment. 

•  Focused  on  commonalities  of  the  uniformed  health  care  system. 

•  Did  not  create  artificial  requirements. 
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•  Had  functional  representatives  on  team. 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS 
and  the  DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM 
program? 

•  Flatten  the  organization,  reduce  the  amount  of  layers  and  bureaucracy  for 
decisions  to  be  made. 

•  Develop  meetings  with  a  purpose. 

•  Create  relationships  with  requirements  community. 

•  Develop  requirements  that  are  static  and  stick  to  it.  Allow  for  flexible  with 
in  the  contract. 

•  Grade  industry  on  RFP  sections  F  and  M  and  not  the  entire  PWS. 

•  Feverage  market  research  that  had  previous  been  done  on  the  commercial 
product. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team, 
what  are  the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of 
other  major  software  acquisition  programs? 

•  Setup  a  Health  SYSCOM  (System  and  Material  Command). 

•  A  Health  SYSCOM  will  have  acquisition  professionals  which  can  rapidly 
capture  technological  advancements  in  a  dynamic  field. 

•  Solidify  what  is  the  true  requirement. 

•  Feverage  commonality,  don’t  highlight  differences. 


Program  Manager  2 
June  27,  2016 

Interview  conducted  via  face-to-face 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related 
programs?  What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past 
EHR  programs? 

•  Yes,  during  the  iEHR. 
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•  I  don’t  think  DOD  does  development  of  these  kind  of  software  programs 
very  well.  It  was  good  at  one  time  during  when  we  were  writing  code  but 
now  when  there  are  commercial  products. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

•  From  my  perspective,  when  all  this  started,  it  was  under  the  control  of 
DCMO  and  DHA,  [they  were]  involved  in  the  process.  The  head  of  PEO 
came  onboard  PEO  was  established.  Then  it  became  a  direct  report  to 
USD  (AT&L)  that  is  unlike  anything  else  in  DOD  acquisition.  What  I 
think  it  also  removed  was  DHA  [because,  they  are]  not  an  acquisition 
entity  and  it  is  not  [their]  core  competency  and  they  don’t  do  it  well. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome 
these  obstacles  and  risks? 

•  I  think  the  acquisition  strategy  was  a  tailored  acquisition.  Not  all  the 
normal  documentation  was  required. 

•  There  was  a  great  leadership  emphasis  to  streamline  the  process,  to  get 
this  thing  moving  fast. 

•  Without  the  ability  to  tailor  things,  you  will  get  drown  in  time,  paper 
work,  and  bureaucracy. 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS 
and  the  DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM 
program? 

•  The  original  [PEO  and  DHMSM  heads]  were  very  hands  on  people.  They 
looked  to  engaged  the  functional  community  directly  and  force  people  to 
make  decisions  and  get  on  board. 

•  [To]  force  people  to  have  discussion  on  where  the  fraction  points  were  and 
to  overcome  them.  You  do  that  by  transparency  and  talking  to  people. 

•  Taking  the  best  of  all  that  information  and  educating  what  the  RFP  would 
be  and  critique  would  be.  Taking  all  that  information  and  using  it  all  for 
your  benefit,  I  think  it’s  a  key  piece  to  the  success. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team, 
what  are  the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of 
other  major  software  acquisition  programs? 

•  Great  leadership,  clearly  define  leadership  roles  are  very  important. 
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•  It  needs  to  be  an  acquisition  organization  that  runs  the  organization  and 
not  something  inside  of  the  functional  community,  meaning  that  the 
medical  guy  shouldn’t  be  the  one  in  charge  of  the  acquisition. 

•  The  functional  [community]  need[s]  to  be  clearly  involved  and  serve  in 
that  role.  [They  have]  to  tell  you  no  kidding  [what]  they  really  need  the 
[system]  to  do. 

•  The  functional  [community]  really  needs  to  be  force  to  define  the 
requirements,  the  tighter  you  that,  the  better  you’ll  be  able  to  buy  it  for 
them.  The  better  the  requirements,  the  better  the  acquisitions  will  be. 


Program  Manager  3 
May  24,  2016 

Interview  conducted  via  face-to-face 


Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related 
programs?  What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past 
EHR  programs? 

•  Yes,  for  iEHR  and  Clinical  Information  Technology  Program  Office 
(CITPO). 

•  DOD  is  a  lousy  software  developer. 

•  For  iEHR,  DOD  attempted  to  develop  and  modify  a  product.  They  [DOD] 
are  not  coders  or  developers. 

•  iEHR  requirements  were  too  problematic,  too  big  of  a  product,  two 
different  secretariat  organizations  with  different  process  and  it  was  too  big 
of  a  product.  Which  was  probably  a  non-starter  because  of  the  two 
fundamentally  different  functions.  Since  process  can’t  be  strategized,  it 
can’t  be  implemented. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

•  Congress  hated  us  [DOD],  due  to  the  bad  press  about  the  failed  EHR  way 
ahead  (2008-2009)  program. 

•  DOD  had  spent  over  2.4  billion  dollars  for  iEHR  that  never  materialized 
from  2008-2013. 

•  Lack  of  governance  and  lack  of  requirements. 
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•  No  requirements  to  put  a  contract  action  in  place. 

•  Lack  of  program  office.  Building  a  program  office  while  attempting  to 
start  program  and  build  strategy  for  ACAT  I  program. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome 
these  obstacles  and  risks? 

•  Being  a  direct  report  to  the  USD  (AT&L)  and  USD  Sectary  of  Defense 
and  with  a  lot  of  leadership  interest  help  reduce  road  blocks  for  a  program 
that  did  not  exist.  Helped  removed  all  barriers. 

•  Reduced  risk  when  placed  under  USD  (AT&L)  by  removed  obstacles, 
allow  for  aggressive  strategy,  test  bed  for  new  strategy  (under  USD 
[AT&L]  guidance).  Because  staff  is  unlikely  to  disapprove  previous 
approved  USD  (AT&L)  guidance. 

•  Highly  tailored  five  fundamental  documents  instead  of  the  16  required 
documents  for  ACAT  I  programs  (acquisition  strategy,  acquisition 
baseline,  engineering  master  plan,  test  strategy,  deployment  change 
management  plan).  These  five  documents  allowed  for  streamline  process 
and  in  coordinated  other  strategy  requirements. 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS 
and  the  DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM 
program? 

•  Complete  transparency. 

•  Said  what  we  were  going  to  do  and  we  did  it. 

•  Aggressive  scheduling  with  milestones. 

•  Announced  first  industry  day  and  held  to  the  schedule  and  then  published 
four  other  industry  days  which  followed. 

•  Published  acquisition  plans. 

•  Meeting  with  companies/industry  often. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team, 
what  are  the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of 
other  major  software  acquisition  programs? 

•  Got  to  have 

1.  Good  requirements. 

2.  Achievable  and  obtainable  commercial  contracting. 
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3.  Accept  commercial  product,  stable  governance. 

4.  Clear  requirements  from  functional  and  technical  side  with  enough 
trade  space  to  allow  for  COTS  environment. 

5.  Allow  enough  space  to  leverage  industry,  be  descriptive  enough  - 
to  allow  DOD  to  buy  something. 

6.  Solid  acquisition  strategy  that  everyone  brought  into  such  as  PEO/ 
USD  (AT&L)/services/functional  sponsors. 

7.  Cannot  customize  accepted  commercial  product.  Must  adopt 
COTS  product  with  commercial  capability. 
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APPENDIX  C.  PROGRAM  INTERVIEWS— MANAGERS 


The  following  are  excerpts  from  in-person  and  Email  correspondence  from 
managers.  Personal  identifiable  information  has  been  removed  to  protect  the  subjects’ 
privacy. 


Manager  1 
May  26,  2016 

Interview  conducted  via  face-to-face 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related  programs? 
What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past  EHR 
programs? 

•  Yes,  iEHR  increment  1  and  2. 

•  The  partnership  between  DOD  and  VA  was  not  fully  committed  to  by  both 
parties. 

•  While  the  program  achieved  acquisition  decisions  to  move  forward,  VA 
withdrew  because  they  only  wanted  to  modernize  their  legacy  system  with 
DOD  funding. 

•  Requirements  articulation  with  the  Medical  customer  base  is  difficult. 

•  Lack  of  empowered  central  leadership  with  the  authority  to  make 
decisions  for  the  enterprise. 

•  Too  many  stakeholders  reluctant  to  commit  to  a  specific  way  forward,  but 
also  too  many  who  can  say  no.  Leadership  by  consensus  does  not  work  in 
a  complex  military  machine. 

•  Stakeholders  are  reluctant  to  commit  because  they  might  miss  out  on  the 
latest  technological  offering.  Many  “leaders”  have  established  themselves 
leveraging  technology  locally.  Scaling  a  home  grown  solution  is  often  not 
feasible  or  runs  into  the  same  challenges  as  a  formal  acquisition. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 

DHMSM  PMO  at  the  beginning  of  the  DHMSM  Program? 

•  Lack  of  formal  requirements. 

•  Lack  of  empowered  central  leadership  with  the  authority  to  make  decision 
for  the  enterprise. 
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Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome  these 
obstacles  and  risks? 

•  Third  parties  were  tasked  to  make  it  happen.  Congress  to  SECDEF, 
SECDEF  to  USD(AT&L)  to  the  new  PEO  to  the  new  PM. 

•  Requirements  development  through  draft  RFPs  and  partnership  with 
industry. 

•  “Tailored”  acquisition  within  the  parameters  of  the  DOD  5000.02 
(Authority  to  Proceed  (ATP)  vice  traditional  Milestone  events). 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS  and  the 

DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM  Program? 

•  Partnership  and  guidance  from  USD  (AT&L),  was  they  tailored  the 
acquisition  processes  to  deliver  capability. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team,  what  are 
the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of  other 
major  software  acquisition  programs? 

•  Force  the  organization  to  formalize  requirements,  to  identify  a  central 
leader  who  is  empowered  to  make  decision  for  the  enterprise  -  meaning 
all  of  DOD  and  four  services,  that  health  care  normalize  between  the  four 
service  regardless  of  geographic  location  or  operational  mission,  it’s  about 
time  that  we  normalize  health  care  and  truly  make  it  purple  -  in  sense 
making  it  purple  and  have  influence  about  how  the  title  10  service  execute 
their  mission. 


Manager  2 
May  25,  2016 

Interview  conducted  via  email  exchange 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related  programs? 
What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past  EHR 
programs? 

•  Yes.  I  have  worked  within  the  Tricare  Management  Activity  and  now 
DHA  since  1998. 1  don’t  know  that  I  would  classify  as  failures. 

•  In  my  opinion,  there  were  delays  due  to  the  lack  of  a  single  empowered 
authority.  There  was  not  one  vision  with  one  way  forward.  Each  Service 
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wanted  to  ensure  respective  processes  and  business  rules  were  protected; 
not  compromised.  Each  Service  has  requirements  inherent  to  its  mission. 

•  CHCS  was  an  early  EHR  program  designed  to  assist  the  medical 
community  at  respective  MTFs.  Each  desktop  was  configured  based  on 
need  of  the  respective  Service  and  MTF.  Then  along  comes  CHCS  II 
(subsequently  AHLTA).  Although  an  improvement,  AHLTA  did  not 
provide  meet  everyone’s  expectations.  CHCS  could  not  simply  be 
uploaded  to  the  new  system. 

•  While  AHLTA  represents  the  largest  DOD  EHR  capabilities  in  the  world, 
it  does  not  completely  support  all  defined  current  needs  and  anticipated 
future  needs.  The  existing  capabilities  fail  to  meet  the  vision  of  a 
comprehensive  EHR  and  do  not  effectively  support  the  presidential 
initiatives  for  an  EHR  or  seamless  data  sharing  with  the  necessary 
partners.  The  advancements  in  technology  and  changes  in  the  complexity 
of  medicine,  healthcare  delivery,  dependencies  on  civilian  managed  care, 
data- sharing  with  the  Federal  and  commercial  partners  and  Operational 
Tempo  require  advanced  EHR  capabilities  to  meet  current  and  future 
needs. 

•  In  my  opinion,  the  lack  of  a  single  authoritative  source,  the  design  of  the 
Government  and  Service  Departments  to  have  multiple  layers  of 
governance  processes,  long  lead  times  to  approve/obtain  funding,  sound 
working  relationships  between  the  functional  (requirements)  and  technical 
communities  along  with  the  rotation  of  Service  members  led  to  significant 
delays  or  failures. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  Program? 

•  Another  obstacle  included  securing  funding.  There  are  many  competing 
priorities  for  funding.  The  Government  Program,  Planning,  Budgeting, 
and  Execution  System  (PPBES)  is  designed  to  plan  for  many  years  of 
future  funding  requirements.  This  system  was  designed  based  on  product 
development/manufacturing,  deployment  and  sustainment.  In  this  ever 
changing  world  of  technology,  the  system  is  not  flexible  to  keep  up. 

•  Another  obstacle  is  staffing.  First  of  all,  a  clear  plan  for  staffing  and 
manning  the  projects  and  programs  is  needed.  Then  obtaining  the  right 
talent,  in  a  timely  manner,  to  execute  a  plan  that  is  filled  with  challenges 
ranging  from  long  hiring  time  for  civilians,  Service  members’  rotation 
schedules  and  competing  requirements  like  the  war,  cost  of  contract 
support,  and  length  of  time  to  award  contract  are  clearly  obstacles  in  being 
able  to  accomplish  the  work. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome  these 
obstacles  and  risks? 
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•  From  my  perspective,  the  biggest  obstacle  was  overcome  when  the  PEO 
DHMS  was  stood  up  under  USD  (AT&L).  This  provided  for  an 
empowered  authoritative  source. 

•  Staffing  obstacles  were  overcome  with  the  agreement  between  DHA  and 
SPAWAR  -  allowing  the  PEO  DHMS  to  hire  skilled  staff  from  the 
Working  Capital  Funds. 

•  With  the  designation  of  . . .  [name  redacted]  as  the  PEO,  [name  redacted] 
rebuilt  the  credibility  at  the  highest  levels,  Congress,  AT&L,  OMB,  and 
the  Services.  This  enabled  the  program  to  secure  funding. 

•  Rebuilding  the  broken  relationships  and  ensuring  the  cost  assessment  and 
program  evaluation  was  included  as  part  of  the  process,  enabled  the  Life 
Cycle  Cost  Estimate  to  be  successfully  developed/implemented. 
Additionally,  via  a  “tailored”  acquisition  approach,  all  stakeholders  were 
part  of  the  process  and  included  in  the  development  of  the  DHMSM 
Acquisition  Strategy 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS  and  the 
DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM  Program? 

•  The  years  of  acquisition  experience  enabled  leadership  to  identify  the 
processes  inherently  required  for  a  successful  COTS  acquisition.  They 
identified  the  right  skills  sets  and  staff  needed  to  meeting  the  challenges  of 
developing  an  RFP  that  clearly  addressed  and  identified  the  requirements 
required  for  a  vendor  to  provide  a  COTS  solution. 

•  They  identified  the  right  skills  sets  and  staff  needed  to  meeting  the 
challenges  of  developing  an  RFP  that  clearly  addressed  and  identified  the 
requirements  required  for  a  vendor  to  provide  a  COTS  solution. 

•  Additionally,  through  their  leadership,  they  ensured  the  right  communities 
were  part  of  the  evaluation  process  to  ensure  the  functional  requirements 
would  be  met.  In  addition  to  their  leadership,  folks  were  highly  motivated 
to  support  the  mission  of  obtaining  a  cradle-to-grave  EHR  for  our  Service 
Members,  Families  and  Beneficiaries. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team,  what  are 
the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of  other 
major  software  acquisition  programs? 

•  Teamwork  and  Communication 

•  Clear  understanding  of  requirements  and  desired  outcomes 

•  Identify  stakeholders  and  beneficiaries  -  make  them  part  of  the  process 

•  Create  positive  working  environments  that  motivate  employees 
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•  Establish  realistic  goals  and  timelines 

Manager  3 
May  31,  2016 

Interview  conducted  via  face-to-face 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related  programs? 
What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past  EHR 
programs? 

•  Yes. 

•  This  was  a  time  when  DOD  was  not  ready  to  do  this  but  we  still  went 

through  a  whole  process  with  the  DHMSM.  The  IEHR  was  trying  to  do 

that  with  us  and  do  it  with  the  VA,  and  VA  was  less  prepared  to  deal  with 
standardization  and  process  flows  than  then  DOD  was.  The  DOD  had 
three  services  Navy,  Army,  Air  Force,  so  the  DOD  has  three  different 
policy  things.  VA  has  120  modules  of  VISTA.  They  have  to  decentralize 
what  they  do,  they  have  medical  centers  that  run  their  own  VISTA  and  has 
its  own  work  flows  i.e.,  billing.  Before  DOD  and  VA  can  work  together, 
you  have  to  have  three  [services]  in  DOD  and  120  [modules]  on  VA.  If 
you  can’t  get  together  what  system  your  building  and  variables  that  you 
will  put  in  play,  then  you’re  sunk. 

•  There  was  a  huge  group  at  the  VA  that  wanted  to  re-litigate  the  decision 
for  the  DOD  not  to  buy  into  the  VISTA  system.  The  VA  decision  was,  that 
we  think  we  should  use  Vista  as  the  basis  for  the  EHR,  DOD  that  doesn’t 
make  sense  to  us,  we  don’t  have  Vista  programmers  and  will  cost  us  more 
to  use  VISTA.  The  VA  recognize  that  DOD  as  a  partner  and  a  cash  source. 
The  smaller  department  will  leverage  the  bigger  department,  so  they  can 
pay  for  it,  is  what  VA  wanted.  You  have  backers  of  VA  that  DOD  is 
leaving  VA  out.  Some  didn’t  understand  interoperability  and  some  knew  it 
but  understood  [that]  VA  would  be  better  with  DOD. 

•  When  VISTA  came  out,  it  was  state  of  the  art  but  industry  has  past  that. 

•  The  DOD  came  to  the  conclusion  earlier  than  the  VA,  because  the  DOD 
experience  with  buying  all  kinds  of  other  weapons  systems.  We  don’t 
design  airplane  anymore,  we  buy  planes  from  Lockheed  and  Boeing 
designs.  There  was  a  time  when  we  designed  our  own  airplanes  and  over 
saw  the  building  of  them.  We  outsource  those  professionals  decades  ago. 
And  the  VA  still  have  VISTA  programmers  on  its  staff. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 

DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 
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•  When  DHMSM  was  created,  you  had  a  medical  community  that  felt  like 
they  have  their  best  and  brightest  in  endeavor  taken  away  from  them  by 
failures  that  happen  in  the  past.  And  by  letting  the  acquisition  people 
procure  the  EHR — what  do  they  know  about  EHR?  There  were  hurt 
feelings  throughout  the  medical  community.  That  was  an  obstacle. 

•  Services  fighting  one  another  and  fighting  with  DHA  to  have  some  of 
governance  and  decision  making  process.  You  had  three  medical  systems 
that  operated  with  autonomy  for  decades  and  now  they  are  being  forced 
into  one  central  organization. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome  these 
obstacles  and  risks? 

•  [The  Head  of  PEO]  said  that  he  was  in  the  DOD  but  not  in  the  medical 
community.  The  DOD  is  in  the  buying  business.  Our  focus  is  not  as  a 
health  care  organization  and  we  need  to  leave  the  best  practices  of  the 
health  care  organization  to  the  industry.  We  need  to  decide  if  we  can  pick 
up  those  best  practices  so  we  don’t  have  to  be  responsible  for  creating 
them. 

•  Taking  away  the  personal  ownership  of  those  most  aligned  with  that 
system,  that  lead  to  a  better  decision. 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS  and  the 
DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM  program? 

•  N/A 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team,  what  are 
the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of  other 
major  software  acquisition  programs? 

•  We  need  to  have  a  medical  health  system  that  is  able  to  leverage  the  best 
practices  of  the  medical  community.  And  adopt  their  best  practice  so  the 
medical  community  can  be  at  the  forefront  of  the  industry.  DOD  made  a 
transition  by  bring  in  an  acquisition  force  in  that  wanted  to  acquire  or  get 
the  best  electronic  health  record,  it  takes  all  the  personal  feeling  out  of  it. 

•  Admin  and  bureaucracy  and  policy  vice  technology  and  that’s  [the]  best 
thing  that  you  can  do  for  any  IT  or  weapons  system  is  [to]  ensure  that  the 
customer  (biggest  stake)  is  removed  from  the  acquisition  decision  and 
leave  the  business  decision  to  acquisition  people. 


56 


Manager  4 
May  17,  2016 

Interview  conducted  via  face-to-face 

Question  1)  Were  you  involved  in  supporting  any  previous  DOD  EHR  related  programs? 
What  do  you  believe  were  some  of  the  key  reasons  for  the  failure  of  these  past  EHR 
programs? 

•  I  was  not  involved  in  previous  programs. 

Question  2)  What  were  the  key  obstacles  and  risks  faced  by  PEO  DHMS  and  the 
DHMSM  PMO  at  the  beginning  of  the  DHMSM  program? 

•  I  believe  that  perceptions  from  the  previous  implementations  affected  the 
program  because  ...  no  one  took  the  project  seriously  to  get  a  contract  in 
place. 

•  It  required  collaboration  across  multiple  organizations  which  requires 
attention  to  ensure  collaboration. 

•  There  were  aggressive  schedule  constraints  due  in  part  to  non-execution  of 
previous  programs. 

Question  3)  What  major  acquisition  strategies  were  implemented  to  overcome  these 
obstacles  and  risks? 

•  Not  sure  it  is  an  acquisition  strategy  but  the  use  of  common  sense  to  not 
allow  stupid  things  get  in  the  way. 

•  Leadership  not  afraid  to  challenge  status  quo  to  move  things  forward  if 
there  is  a  road  block. 

Question  4)  How  would  you  describe  the  managerial  strategies  of  PEO  DHMS  and  the 
DHMSM  PMO,  and  how  did  they  contribute  to  the  success  of  the  DHMSM  Program? 

•  They  weren’t  afraid  to  take  risks. 

•  They  did  not  wait  to  have  100  percent  of  items  complete  before  they 
would  execute.  For  example,  as  information  became  available,  [the]  draft 
RFP  were  released  on  multiple  occasions  to  get  industry  involved  early. 

Question  5)  Based  on  your  experiences  as  a  member  of  the  PEO  DHMS  team,  what  are 

the  key  lessons  learned  and  advice  you  can  share  that  can  benefit  DOD  PM  of  other 

major  software  acquisition  programs? 
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•  Do  not  be  afraid  to  take  risks. 

•  Encourage  collaboration  to  include  frequent  and  early  interaction. 

•  Do  not  be  afraid  to  ask  or  recommend  to  streamlining  the  acquisition 
process. 
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